Senior Data Engineer [with Databricks] – LATAM

LATAM

About Distillery

Distillery Tech Inc accelerates innovation through an unyielding approach to nearshore software development. The world’s most innovative technology teams choose Distillery to help accelerate strategic innovation, fill a pressing technology gap, and hit mission-critical deadlines. We support essential applications, mobile apps, websites, and eCommerce platforms by placing senior, strategic technical leaders and deploying fully managed technology teams that work intimately alongside our client’s in-house development teams. At Distillery Tech Inc, we’re not here to reinvent nearshore software development; we’re on a mission to perfect it.



Distillery Tech Inc is committed to diversity and inclusion. We actively seek to cultivate a workforce that reflects the rich tapestry of perspectives, backgrounds, and experiences present in our society. Our recruitment efforts are dedicated to promoting equal opportunities for all candidates, regardless of race, ethnicity, gender, sexual orientation, disability, age, or any other dimension of diversity.


About the Position

We are seeking a proficient Data Engineer with expertise in Databricks to join our dynamic team. The ideal candidate will possess a strong technical background in building, managing, and optimizing data systems while leveraging the Databricks platform. This role requires a combination of technical prowess, innovative problem-solving skills, and a collaborative mindset to support data-driven decision-making processes for our clients.


Responsibilities

  • Design, implement, and maintain data pipelines and workflows using Databricks and other tools.
  • Develop scalable data warehouse and data lake solutions leveraging the Databricks platform.
  • Collaborate with data scientists and analysts to support data modeling and machine learning initiatives.
  • Optimize data systems and processes for performance, scalability, and reliability.
  • Work with the Databricks/Spark ecosystem to process large-scale datasets efficiently.
  • Utilize tools like Polars and Pandas for data manipulation and transformation.
  • Implement data engineering best practices and frameworks to ensure high-quality data solutions.
  • Engage with stakeholders to understand data requirements and deliver solutions that meet business needs.


Requirements

  • Proven experience with Databricks and its ecosystem.
  • Expertise in building data warehouse and data lake solutions using Databricks.
  • Proficiency in Python, including libraries like Polars and Pandas.
  • Hands-on experience with the Databricks/Spark ecosystem and related libraries (e.g., scikit-learn, LightGBM, GLM).
  • Strong knowledge of data engineering frameworks and best practices.
  • Exceptional SQL proficiency.
  • Medium to strong expertise with one or two major cloud platforms, preferably Azure.
  • A preference for leveraging existing solutions and tools over introducing third-party tools unless necessary.

Nice To Have

  • Experience with Oracle and Progress databases.
  • Familiarity with SDKs from Azure ML, Databricks, and other data science/machine learning ecosystems.
  • Expertise in API technologies such as GraphQL and gRPC.

Why You'll Like Working Here

Join a global team committed to Distillery's core values: Unyielding Commitment, Relentless Pursuit, Courageous Ambition, and Authentic Connection.

  • 100% Remote Work: Enjoy the freedom to work from anywhere while collaborating with a diverse, multinational team.
  • Competitive Compensation: Generous and competitive package in USD, along with a comprehensive benefits plan.
  • Flexible Hours: Create a schedule that aligns with your life and priorities.
  • Home Office Setup: Receive all the hardware and software needed to succeed from home.
  • Innovative Workplace: Collaborate with the global Top 1% of talent in a multicultural and dynamic environment.