Senior Data Engineer
1 month ago
This is a remote position.
We are seeking a highly skilled Senior Spark Engineer with expertise in Python, Spark, Databricks, and AWS to join our dynamic team. The ideal candidate will possess a deep understanding of Spark architecture and be proficient in fine-tuning Spark jobs for optimal performance. Additionally, strong knowledge of software engineering best practices, DevOps principles including CI/CD pipelines, Kubernetes, and either Jenkins, Airflow, or SageMaker, is essential for this role. Experience in MLOps is desirable but not mandatory.
Requirements
- Proficiency in Spark, Python, Databricks, and AWS services.
- 7-10 years of experience
- Lead the design, development, and implementation of Spark-based solutions for complex data processing tasks.
- Collaborate with cross-functional teams to gather requirements and translate them into technical specifications.
- Implement best practices in software engineering, DevOps, and MLOps methodologies.
- Design and maintain data pipelines for efficient data processing and analysis.
- Perform data modeling and schema design to support business requirements.
- Mentor junior team members and provide technical guidance as needed.
- Stay updated with the latest advancements in Spark, Python, and related technologies, and advocate for their adoption when appropriate.
- Bachelor's degree in Computer Science, Engineering, or a related field. Master's degree preferred.
- Strong understanding of software engineering principles and best practices.
- Hands-on experience with DevOps tools and practices including CI/CD pipelines, Kubernetes, and either Jenkins, Airflow, or SageMaker.
- Experience with MLOps practices is a plus.
- Solid understanding of data engineering concepts and data modeling techniques.
- Excellent problem-solving skills and attention to detail.
- Ability to work independently and lead tasks effectively.
- Strong communication and interpersonal skills.
Requirements
Proficiency in Spark, Python, Databricks, and AWS services. 7-10 years of experience Lead the design, development, and implementation of Spark-based solutions for complex data processing tasks. Collaborate with cross-functional teams to gather requirements and translate them into technical specifications. Implement best practices in software engineering, DevOps, and MLOps methodologies. Design and maintain data pipelines for efficient data processing and analysis. Perform data modeling and schema design to support business requirements. Mentor junior team members and provide technical guidance as needed. Stay updated with the latest advancements in Spark, Python, and related technologies, and advocate for their adoption when appropriate. Bachelor's degree in Computer Science, Engineering, or a related field. Master's degree preferred. Strong understanding of software engineering principles and best practices. Hands-on experience with DevOps tools and practices including CI/CD pipelines, Kubernetes, and either Jenkins, Airflow, or SageMaker. Experience with MLOps practices is a plus. Solid understanding of data engineering concepts and data modeling techniques. Excellent problem-solving skills and attention to detail. Ability to work independently and lead tasks effectively. Strong communication and interpersonal skills.
-
Senior Data Engineer
7 days ago
India hunterAI Full timeCompany Description Join hunterAI, a leading provider of advanced data-driven solutions, specializing in OCR, ML, Python, MySQL, Data Analysis, and NLP. As a remote-first company, we foster a collaborative and inclusive work environment, prioritizing work-life balance. Explore how we leverage cutting-edge technologies to drive business success through data...
-
Senior Data Engineer
7 days ago
India hunterAI Full timeCompany DescriptionJoin hunterAI, a leading provider of advanced data-driven solutions, specializing in OCR, ML, Python, MySQL, Data Analysis, and NLP. As a remote-first company, we foster a collaborative and inclusive work environment, prioritizing work-life balance. Explore how we leverage cutting-edge technologies to drive business success through data...
-
Senior Data Engineer
1 week ago
India Accelerant Full timeLocation- IndiaExperience - 8+ yearsRole DescriptionThis is a remote contract role for a Senior Data Engineer at Accelerant. As a Senior Data Engineer, you will be responsible for the day-to-day tasks associated with data engineering, including data modeling, ETL (Extract Transform Load), data warehousing, and data analytics. You will work remotely and...
-
Senior Data Engineer
1 week ago
India Accelerant Full timeLocation- India Experience - 8+ years Role Description This is a remote contract role for a Senior Data Engineer at Accelerant. As a Senior Data Engineer, you will be responsible for the day-to-day tasks associated with data engineering, including data modeling, ETL (Extract Transform Load), data warehousing, and data analytics. You will work remotely...
-
Senior Machine Learning Engineer
2 weeks ago
india K2 Data & Analytics Full timeJob Description : We are on the lookout for an enthusiastic ML Engineer with over five years of experience to be part of our team, which is focused on developing cutting-edge Video Ad Data technologies. The perfect fit for this role would be someone with a solid grasp of machine learning algorithms and a background in designing and executing large-scale...
-
Senior Data Engineer
5 days ago
india Next-Link Full timeJob Description We are searching for a talented Senior Data Engineer to join our team and support our client, a leading pharmaceutical company committed to pioneering healthcare solutions for both humans and animals. In this exciting role, you'll play a pivotal role in scaling up and globally rolling out their Customer Data Platform (CDP), a critical...
-
Senior Data Engineer
1 day ago
india INI8 LABS Full timeJob Description: We are looking for an experienced Senior Azure Data Engineer to join our team. The successful candidate should have experience working with Data Architects, Analyst and Stakeholders to develop, maintain and test data platforms. Experience: Minimum 4+ yrs Experience with Azure(Azure data bricks, Azure data factory, PySpark) minimum of 4...
-
Senior Data Engineer
6 hours ago
India SourceBae Full timeSenior Data EngineerExp-6+YrsShift Time : 2 PM - 11 PMAs a Senior Data Engineer , you will: Design, Architect, and Develop solutions leveraging cloud big data technology to ingest,process and analyze large, disparate data sets to exceed business requirements Develop systems that ingest, cleanse and normalize diverse datasets, develop datapipelines from...
-
Senior Data Engineer
4 weeks ago
india Recro Full timePosition: Senior Data Engineer Location: Bangalore Type: Full-Time Experience Level: 4.5+ years Job Description: We're building a high-performing data team and searching for a skilled and passionate Data Engineer to join us! You'll be responsible for designing, developing, and maintaining our data infrastructure, ensuring a steady flow of clean,...
-
Senior Data Engineer
1 week ago
india V3 Staffing Solutions India P Limited Full timeWe have this opening with the global company that is shaping the future of gaming through groundbreaking technology and unparalleled creativity. Senior Data Engineer (AWS): Crafting Efficient Data Pipelines for Cloud Platforms We're seeking a seasoned Senior Data Engineer with over 9 years of hands-on experience in crafting and implementing highly...
-
Senior Data Engineer
2 weeks ago
india Tredence Inc. Full timeData Engineering - Associate Manager (6-10 Years) / Manager (10-14 Years) This position requires someone with good problem solving, business understanding and client presence. Overall professional experience of the candidate should be above 10 years. A minimum of 6 years of experience in leading and managing a client portfolio in Data Engineering space....
-
CTO (Senior Data Engineer)
1 month ago
India SHMTH Capital Full timeAt SHMTH Capital, we are on a mission to bridge the gap between tradition and innovation in finance. We offer a transparent, decentralized, and institutional-grade digital asset management platform accessible to all, with the goal of elevating financial growth in the dynamic world of cryptocurrency. Role Description Embark on an exhilarating journey as a...
-
CTO (Senior Data Engineer)
1 month ago
India SHMTH Capital Full timeAt SHMTH Capital, we are on a mission to bridge the gap between tradition and innovation in finance. We offer a transparent, decentralized, and institutional-grade digital asset management platform accessible to all, with the goal of elevating financial growth in the dynamic world of cryptocurrency.Role DescriptionEmbark on an exhilarating journey as a...
-
Sr Specialist, Data Solutions
7 hours ago
India Liberty Data Analytics Full timeSenior Specialist, Data Solutions Provides, builds, and maintains data solutions to meet the needs of the enterprise. Helps ensure those solutions can be scaled appropriately. Under minimal to no direction, works within broad limits and authority on highly complex assignments requiring specialized knowledge. Viewed as senior resource in given field for...
-
Sr Specialist, Data Solutions
1 day ago
india Liberty Data Analytics Full timeSenior Specialist, Data Solutions Provides, builds, and maintains data solutions to meet the needs of the enterprise. Helps ensure those solutions can be scaled appropriately. Under minimal to no direction, works within broad limits and authority on highly complex assignments requiring specialized knowledge. Viewed as senior resource in given field for...
-
Senior Data Engineer
2 days ago
india Infinite Commerce Full timeNeiron, in collaboration with Infinite Commerce, is looking to fill a Senior Data Engineer based in India. This position will support the Infinite Commerce business, but will be employed directly by Neiron. Infinite Commerce works with Neiron as our local representative to support local employment and ensure a world class employment experience for our lndia...
-
Senior Data Engineer
2 days ago
India Infinite Commerce Full timeNeiron, in collaboration with Infinite Commerce, is looking to fill a Senior Data Engineer based in India. This position will support the Infinite Commerce business, but will be employed directly by Neiron. Infinite Commerce works with Neiron as our local representative to support local employment and ensure a world class employment experience for our lndia...
-
Senior Data Engineer
1 week ago
india Next-Link Full timeJob Description Senior Data Engineer - Brussels - 12 moisOne of our important clients is currently looking for a Senior Data Engineer for a 12-month contract in Brussels.We are looking for a Senior Data Engineer who has extensive experience in developing and deploying data pipelines, infrastructure, and models for our data science projects. You will work...
-
Senior Data Engineer
3 weeks ago
india Cayian India Pvt Limited Full timeJob Description Cayian is a global IT and business service provider that delivers consultancy, capability, and system integration services to our partners. Our unique partner model is optimised to accelerate, continuously improve quality, and drive business growth. Find out more about us here. The role As a Senior Data Engineer, you will be...
-
Senior Data Engineer
3 weeks ago
india Circuitry.ai Full timeJob Description: Location: Hyderabad, India - Hybrid Remote (3 days a week in office preferred) Position Overview: As a Data Engineer focused on AI development, you will play a pivotal role in the design, development, and deployment of data infrastructure and pipelines that support artificial intelligence and machine learning initiatives within...