Data Engineer for Scalable Data Pipelines
2 weeks ago
We are seeking an experienced Data Engineer to join our team at Colan Infotech Private Limited. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining scalable data pipelines using Pyspark / Scala Spark, Databricks, Python, and SQL.
The ideal candidate will have strong programming skills with the ability to write optimized and reusable high-quality code. You will work closely with cross-functional teams to understand business requirements and collaborate with them to implement best practices for data pipelines optimization, ETL, and query performance.
In this role, you will also be responsible for troubleshooting and resolving issues in data pipelines, ensuring minimal downtime. Additionally, you will create and maintain documentation for data pipelines and workflows.
Key Responsibilities:
- Design and develop scalable data pipelines using Pyspark / Scala Spark, Databricks, Python, and SQL
- Optimize SQL queries for efficient data extraction and manipulation
- Demonstrate expertise in Databricks including workflow management and data exploration
- Collaborate with cross-functional teams to understand business requirements
- Implement best practices for data pipelines optimization, ETL, and query performance
- Troubleshoot and resolve issues in data pipelines, ensuring minimal downtime
- Create and maintain documentation for data pipelines and workflows
Requirements:
- Pyspark / Scala Spark experience
- Databricks experience
- Python experience
- SQL experience
- Strong programming skills
- Able to write optimized and reusable high-quality code
Salary: $120,000 - $180,000 per year
Benefits: We offer a comprehensive benefits package, including health insurance, retirement plan, and paid time off.
-
Senior Data Engineer
4 weeks ago
Chennai, Tamil Nadu, India Wavicle Data Solutions Full timeWe are seeking an Experienced AWS Data Architect with a strong background in designing, building, and maintaining scalable data pipelines and solutions on AWS. The ideal candidate will have a deep understanding of AWS services, data engineering best practices, and cloud architecture.Key Responsibilities:Design, develop, and maintain robust ETL pipelines on...
-
Chief Data Science Engineer
3 weeks ago
Chennai, Tamil Nadu, India NTT DATA Full timeJob SummaryWe are seeking an experienced Chief Data Science Engineer to join our team at NTT DATA. As a key member of our organization, you will be responsible for designing and implementing machine learning systems, researching and developing ML algorithms and tools, and selecting appropriate data sets.About the RoleYou will work closely with...
-
Data Engineer for Scalable Data Solutions
1 week ago
Chennai, Tamil Nadu, India Truetech Full timeWe are seeking an experienced Data Engineer to join our team at Truetech.**Job Summary:**We are looking for a skilled Big Data Engineer to design, build, and optimize data pipelines using technologies like Apache Spark, Python, Apache Airflow, Kafka, and GCP (BigQuery, DataProc, Cloud Composer). You will work on scalable data solutions, real-time streaming,...
-
Data Engineer for Scalable Data Solutions
1 month ago
Chennai, Tamil Nadu, India m2p fintech Full time**Job Description**We are seeking a skilled Data Engineer to join our team at M2P Fintech. As a Data Engineer, you will play a key role in building and maintaining scalable data solutions that drive business growth.**Responsibilities**Develop and maintain highly scalable data processing applications.Write efficient, reusable, and well-documented code.Deliver...
-
Data Pipeline Architect
2 weeks ago
Chennai, Tamil Nadu, India Colan Infotech Private Limited Full timeGreetings from Colan Infotech Private LimitedAs a Senior Data Engineer, you will be responsible for designing and developing scalable data pipelines using Pyspark / Scala Spark, Azure Databricks, Azure Data Factory, Azure Datalake, Python, and SQL.We are looking for an experienced professional who can optimize SQL queries for efficient data extraction and...
-
Engineering Team Lead
2 weeks ago
Chennai, Tamil Nadu, India UnitedHealth Group Full timeAt UnitedHealth Group, we're committed to helping people live healthier lives and making the health system work better for everyone.We're seeking an experienced Engineering Team Lead - Data Pipelines to join our team. This role is responsible for leading a team of engineers in designing, developing, and maintaining highly scalable and reliable data...
-
Data Pipeline Architect
1 week ago
Chennai, Tamil Nadu, India Walking Tree Full timeJob Title: Data Pipeline Architect">About the Role:">We are seeking an experienced ETL Test Engineer to join our team at Walking Tree. As a key member of our data engineering team, you will design, implement and continuously expand data pipelines by performing extraction, transformation, and loading activities.">Your Responsibilities:">">Gather requirements...
-
Data Engineer for Scalable Data Solutions
2 days ago
Chennai, Tamil Nadu, India m2p fintech Full timeJob Description:M2P Fintech is looking for a skilled Data Engineer to build and set up advanced data solutions. As part of a collaborative cross-functional team, you will work in a fast-paced and agile work environment, delivering high-quality and innovative solutions that have a significant business impact.Key Responsibilities: - Develop and maintain highly...
-
GCP Data Engineer
1 month ago
Chennai, Tamil Nadu, India Saaki Argus & Averil Consulting Full timeAbout Saaki Argus & Averil Consulting:We are a key player in the electric vehicle industry, specializing in delivering sustainable mobility solutions for the commercial transportation sector.Our team is seeking a skilled GCP Data Engineer to join us in Chennai.The ideal candidate will have 7 to 12 years of experience in designing, developing, and maintaining...
-
Senior Cloud Data Engineer
2 days ago
Chennai, Tamil Nadu, India System Soft Technologies Full timeJob OverviewSystem Soft Technologies is a premier technology company providing exceptional consulting services and solutions that drive innovation, enhance business value, and boost competitiveness. With over 25 years of experience in building trusted partnerships with clients, we have grown into a leading enterprise with a collective resources of over 1,200...
-
Data Engineering Expert
2 days ago
Chennai, Tamil Nadu, India Tata Consultancy Services Full timeAbout This RoleWe are seeking an experienced Data Engineer to join our team at Tata Consultancy Services. As a Data Engineer, you will be responsible for designing and developing scalable data pipelines and data lakes/warehouses in a cloud environment, preferably Azure.ResponsibilitiesBuilding and optimizing data pipelines and data lakes/warehouses in a...
-
Chennai, Tamil Nadu, India Mastech InfoTrellis Full timeMastech InfoTrellis seeks a seasoned Data Engineer to design and implement large-scale data pipelines on Google Cloud Platform (GCP).Estimated Salary: $120,000 - $180,000 per year.About the Role:We are looking for an experienced professional with expertise in GCP services, particularly BigQuery, Dataflow, Pub/Sub, and Cloud Storage. The ideal candidate will...
-
Data Pipeline Architect
2 weeks ago
Chennai, Tamil Nadu, India Hirexa Solutions Full timeJob OverviewHirexa Solutions is seeking a highly skilled Data Pipeline Architect to design and implement efficient data pipelines using Scala and Spark.About the RoleThis is an exceptional opportunity for experienced professionals with strong expertise in big data technologies, scalable data processing, and API development.The ideal candidate will have 6+...
-
Data Engineer for Scalable Big Data Solutions
2 weeks ago
Chennai, Tamil Nadu, India RiskInsight Consulting Pvt Ltd Full timeWe are seeking an experienced Data Engineer to join our team at RiskInsight Consulting Pvt Ltd. This role involves designing, developing, and deploying scalable Big Data applications using Apache Spark.ResponsibilitiesCollaborate with data scientists and business analysts to understand requirements and translate them into technical solutions.Write efficient...
-
Chennai, Tamil Nadu, India SM IT SERVICES Full timeAbout the RoleWe are seeking a highly skilled Data Engineer to join SM IT SERVICES. The ideal candidate will have a strong background in data engineering and a deep understanding of Google Cloud Platform (GCP) services.Key Responsibilities:Design, develop, and maintain scalable and robust data pipelines using GCP services.Work with BigQuery to design and...
-
Cloud Engineering Specialist
6 days ago
Chennai, Tamil Nadu, India Prodapt Full timeAbout Prodapt: As a leading consulting and managed services provider, we empower our clients to excel in the digital era.Job Summary: We are seeking an experienced Cloud Engineering Specialist to design, develop, and deploy robust data pipelines using Google Cloud Platform (GCP) services. This role involves collaborating with cross-functional teams to...
-
Senior Machine Learning Engineer
3 days ago
Chennai, Tamil Nadu, India Crayon Data Full timeWe are seeking a highly skilled Senior Machine Learning Engineer to join our team in Chennai. This is a fantastic opportunity to work on large scale data solutions using machine learning, operation research and pattern searching methods.About Crayon DataCrayon Data is a leading provider of AI-led revenue acceleration solutions, headquartered in Singapore...
-
AWS Data Engineering Lead
6 days ago
Chennai, Tamil Nadu, India Tata Consultancy Services Full timeTata Consultancy Services is seeking an experienced AWS Data Engineer Lead to join our team. This is a challenging role that requires strong technical expertise in designing and implementing scalable, secure, and efficient data architectures using AWS services.About the RoleIn this role, you will be responsible for leading the design, implementation, and...
-
Cybersecurity Engineer for NTT DATA
3 weeks ago
Chennai, Tamil Nadu, India NTT DATA Full timeCompany OverviewNTT DATA is a $30+ billion trusted global innovator of business and technology services. We serve 75% of the Fortune Global 100 and are committed to helping clients innovate, optimize and transform for long-term success.We invest over $3.6 billion each year in R&D to help organizations and society move confidently and sustainably into the...
-
Senior Data Engineer
2 weeks ago
Chennai, Tamil Nadu, India American Express Full timeData Engineer I Job DescriptionCompany OverviewAmerican Express is a global leader in the financial services industry, providing innovative payment solutions to customers around the world. Our company values diversity, equity, and inclusion, and we strive to create an environment where every colleague feels valued and empowered to succeed.Estimated Salary...