Senior Data Engineer(Databricks)

Design, develop, and maintain data pipelines using Databricks and Spark, and other cloud technologies as needed
Optimize data pipelines for performance, scalability, and reliability
Ensure data quality and integrity throughout the data lifecycle
Collaborate with data scientists, analysts, and other stakeholders to understand and meet their data needs
Troubleshoot and resolve data-related issues, and provide root cause analysis and recommendations
Document data pipeline specifications, requirements, and enhancements, and communicate them effectively to the team and management
Create new data validation methods and data analysis tools, and share best practices and learnings with the data engineering community
Implement ETL processes and data warehouse solutions, and ensure compliance with data governance and security policies
 

Please Note: This is a hybrid position and will require at least 2 days in the office per week. Successful candidates will need to complete a background check.

Requirements

Qualifications

Bachelor’s degree in Computer Science, Engineering, or related field, or equivalent work experience
5+ years of experience in data engineering with Databricks and Spark
Proficient in SQL and Python, and familiar with Java or Scala
Experience with cloud platforms, such as Azure or AWS
Experience with big data technologies, such as Kafka, Hadoop, Hive, etc.
Experience with data warehouse and data lake concepts and architectures
Experience with data integration and ETL tools, such as Azure Data Factory or Talend
Experience with data visualization and reporting tools, such as Power BI or Tableau
Strong analytical and problem-solving skills
Excellent communication and teamwork skills
 

Benefits

Benefits

We are a collective group of people and collaboration is key to our process
We don’t work for our clients, we work with them
A Flexible Hybrid Working Environment
Easily accessible downtown location
Competitive compensation commensurate with experience
Everyone brings something valuable to the table in our supportive, challenging, and collaborative, diverse work environment

All Employees Can Participate in:

Company paid health benefits: 100% medical, dental and vision coverage
Corporate-discounted Gym Membership through GoodLife
Company discount program including Travel, Shopping, Attractions, Wellness, & Sporting events, just to name a few
Access to an Employee & Family Assistance program (EAP)
Employee Referral Program
Employee Opportunity Program
Professional Development Program
Philanthropic Events
Social Events

Inclusion & Diversity

We foster an inclusive and diverse workforce, believing our strength stems from our individual differences. Our employees, partners, and clients continuously benefit from the innovation and creativity grounded in these values. We strive to be a company that attracts a diverse group of highly skilled people who know that their contributions will be valued and that they will be heard. We are committed to building a corporate culture with people who are excited to join our team, do their best work, and grow with us!