Freelance Data Engineer

3 weeks ago

Moradabad, India upGrad Full time

We are seeking a highly skilled and motivated Data Engineer to join our team. The ideal candidate will be responsible for designing, developing, and optimizing large-scale data pipelines and data warehouse solutions, utilizing a modern, cloud-native data stack. You'll play a crucial role in transforming raw data into actionable insights, ensuring data quality, and maintaining the infrastructure required for seamless data flow.Key ResponsibilitiesDevelop, construct, test, and maintain robust and scalable large-scale ETL pipelines using PySpark for processing and Apache Airflow for workflow orchestration.Design and implement both Batch ETL and Streaming ETL processes to handle various data ingestion requirements.Build and optimize data structures and schemas in cloud data warehouses like AWS Redshift.Work extensively with AWS data services, including AWS EMR for big data processing, AWS Glue for serverless ETL, and Amazon S3 for data storage.Implement and manage real-time data ingestion pipelines using technologies like Kafka and Debezium for Change Data Capture (CDC).Interact with and integrate data from various relational and NoSQL databases such as MySQL, PgSQL (PostgreSQL), and MongoDB.Monitor, troubleshoot, and optimize data pipeline performance and reliability.Collaborate with data scientists, analysts, and other engineering teams to understand data needs and deliver high-quality, reliable data solutions.Ensure data governance, security, and quality across all data platforms.Required Skills & QualificationsTechnical SkillsExpert proficiency in developing ETL/ELT solutions using PySpark.Strong experience in workflow management and scheduling tools, specifically Apache Airflow.In-depth knowledge of AWS data services including:AWS EMR (Elastic MapReduce)AWS GlueAWS RedshiftAmazon S3Proven experience implementing and managing data streams using Kafka.Familiarity with Change Data Capture (CDC) tools like Debezium.Hands-on experience with diverse database technologies: MySQL, PgSQL, and MongoDB.Solid understanding of data warehousing concepts, dimensional modeling, and best practices for both batch and real-time data processing.Proficiency in a scripting language, preferably Python.General QualificationsBachelor’s or Master’s degree in Computer Science, Engineering, or a related field.Excellent problem-solving, analytical, and communication skills.Ability to work independently and collaboratively in a fast-paced, dynamic environment.Nice to Have (Preferred Skills)Experience with Infrastructure as Code (e.g., Terraform, CloudFormation).Knowledge of containerization technologies (Docker, Kubernetes).Familiarity with CI/CD pipelines.

  • Freelance Data Engineer

    3 weeks ago

    Moradabad, India upGrad Full time

    We are seeking a highly skilled and motivated Data Engineer to join our team. The ideal candidate will be responsible for designing, developing, and optimizing large-scale data pipelines and data warehouse solutions, utilizing a modern, cloud-native data stack. You'll play a crucial role in transforming raw data into actionable insights, ensuring data...

  • Data Engineer

    3 weeks ago

    Moradabad, India Tata Consultancy Services Full time

    Opening - PUNE/Bangalore/Hyd/ChennaiAbout the RoleRole - Data Engineer - Quantexa ,Big Data Hadoop, Spark and ScalaResponsibilitiesDevelop Spark and SCALA code as per requirement.Build efficient and optimized ETLs for data ingestion/transformations.Should be able to customize and update the Quantexa UI as per business requirement.Documentation of the...

  • Senior data engineer

    2 days ago

    Moradabad, India Egen Full time

    Senior Data Engineer – GCP Job Overview: We are looking for a skilled and motivated Senior Data Engineer with strong experience in Python programming and Google Cloud Platform (GCP) to join our data engineering team. The ideal candidate will be responsible for designing, developing, and maintaining robust and scalable ETL (Extract, Transform, Load) data...

  • Senior data engineer

    2 weeks ago

    Moradabad, India Coffeee.io Full time

    Job Title: Lead Data Engineer – Snap LogicLocation: RemoteExperience: 8+ years (with 3+ years in Snap Logic ETL)Employment Type: Full-timeRole Overview:We are seeking a Lead Data Engineer – Snap Logic to design, build, and optimize data integration pipelines across on-premise and cloud platforms. The ideal candidate will have hands-on experience with...

  • Backend engineer — data

    3 weeks ago

    Moradabad, India Branch International Full time

    Branch OverviewImagine a world where every person has improved access to financial services. People could start new businesses, pay for their children’s education, cover their emergency medical bills – the possibilities to improve life are endless.Branch is a global technology company revolutionizing financial access for millions of underserved banking...

  • Senior GCP Data Engineer

    3 weeks ago

    Moradabad, India Adastra Full time

    Job Description We are seeking a results-driven GCP Data Engineer with a proven track record in architecting and delivering end-to-end data migration solutions. The ideal candidate will have extensive experience migrating complex, large-scale on-premises data systems (Oracle, Teradata, SQL Server, etc.) to Google Cloud Platform (GCP), and optimizing...

  • Data Engineer

    3 weeks ago

    Moradabad, India People Prime Worldwide Full time

    About the Company Our client is a trusted global innovator of IT and business services, present in 50+ countries. They specialize in digital & IT modernization, consulting, managed services, and industry-specific solutions. With a commitment to long-term success, they empower clients and society to move confidently into the digital future. Title: Data...

  • Data Engineering Manager – AI

    2 weeks ago

    Moradabad, India AIMLEAP Full time

    Data Engineering Manager – AI & AutomationExperience: 7 to 12 Years Location: Remote (Work from Home) / Bangalore / India Mode of Engagement: Full-time No of Positions: 2 Educational Qualification: MBA / B.E / B.Tech / M.Tech / MCA / Computer Science / IT Industry: IT / Data / AI / E-commerce / FinTech / Healthcare Notice Period: Immediate  What We...

  • Data Architect

    3 weeks ago

    Moradabad, India People Prime Worldwide Full time

    Important Note (Please Read Before Applying)🚫 Do NOT apply if:You have less than 10 years of experience in data architectureYou do not have hands-on experience in implementing data integrationsYou come from a purely non-architecture background (e.g., only reporting/analyst roles)✅ Apply ONLY if you meet ALL the mandatory requirements listed above....

  • Procurement Data Manager

    2 days ago

    Moradabad, India Asenium Consulting Full time

    For one of my customer we are looking for a Procurement Data consultant with experience in Sievo.Duration: 4+ months(renewable)Location: 100% remote(european time zone)Description:This person will be responsible to maintain existing Sievo reports using procurement data from ERP/SCUPA. They’ll translate numbers into business insights for buyers, not build...