Freelance Data Engineer

3 weeks ago

india, IN upGrad Full time

We are seeking a highly skilled and motivated Data Engineer to join our team. The ideal candidate will be responsible for designing, developing, and optimizing large-scale data pipelines and data warehouse solutions, utilizing a modern, cloud-native data stack. You'll play a crucial role in transforming raw data into actionable insights, ensuring data quality, and maintaining the infrastructure required for seamless data flow.Key ResponsibilitiesDevelop, construct, test, and maintain robust and scalable large-scale ETL pipelines using PySpark for processing and Apache Airflow for workflow orchestration.Design and implement both Batch ETL and Streaming ETL processes to handle various data ingestion requirements.Build and optimize data structures and schemas in cloud data warehouses like AWS Redshift.Work extensively with AWS data services, including AWS EMR for big data processing, AWS Glue for serverless ETL, and Amazon S3 for data storage.Implement and manage real-time data ingestion pipelines using technologies like Kafka and Debezium for Change Data Capture (CDC).Interact with and integrate data from various relational and NoSQL databases such as MySQL, PgSQL (PostgreSQL), and MongoDB.Monitor, troubleshoot, and optimize data pipeline performance and reliability.Collaborate with data scientists, analysts, and other engineering teams to understand data needs and deliver high-quality, reliable data solutions.Ensure data governance, security, and quality across all data platforms.Required Skills & QualificationsTechnical SkillsExpert proficiency in developing ETL/ELT solutions using PySpark.Strong experience in workflow management and scheduling tools, specifically Apache Airflow.In-depth knowledge of AWS data services including:AWS EMR (Elastic MapReduce)AWS GlueAWS RedshiftAmazon S3Proven experience implementing and managing data streams using Kafka.Familiarity with Change Data Capture (CDC) tools like Debezium.Hands-on experience with diverse database technologies: MySQL, PgSQL, and MongoDB.Solid understanding of data warehousing concepts, dimensional modeling, and best practices for both batch and real-time data processing.Proficiency in a scripting language, preferably Python.General QualificationsBachelor’s or Master’s degree in Computer Science, Engineering, or a related field.Excellent problem-solving, analytical, and communication skills.Ability to work independently and collaboratively in a fast-paced, dynamic environment.Nice to Have (Preferred Skills)Experience with Infrastructure as Code (e.g., Terraform, CloudFormation).Knowledge of containerization technologies (Docker, Kubernetes).Familiarity with CI/CD pipelines.

  • Freelance Data Quality Automation

    3 weeks ago

    india, IN Leading MNC Part time

    Looking for a Freelance Data Quality Automation to join a team of rockstar developers. The candidate should have a minimum of 4+ yrs. of experience.There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You will...

  • Data Engineer

    3 weeks ago

    india, IN Dautom Full time

    Job Description: Data Engineer (Big Data/Kafka) We are seeking a highly experienced Senior Data Engineer with a deep background in Big Data technologies to join our team. This is a contract role for a major project in the Banking sector.Key Details:Role: Data EngineerIndustry: Banking (Financial Services)Work Location: Remote (India)Contract Duration: 6...

  • Freelance Agentic AI Engineer

    3 weeks ago

    india, IN CodeVyasa Part time

    Looking for a Freelance Agentic AI Engineer to join a team of rockstar developers. The candidate should have a minimum of 4+ yrs. of experience.There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You will report...

  • Freelance Consultant

    3 weeks ago

    india, IN Taskify Full time

    We're Hiring "Freelance Consultant (Freelance/Remote)" | Earn up to $2500 per monthLocation: India (Remote)What You Will Be Doing:Contribute to training and refining cutting-edge AI systemsAdopt a “user mindset” to produce natural and realistic data for AI developmentDevelop rubrics to structure and address user needs effectivelyEvaluate AI-generated...

  • Freelance Biostatistician

    3 weeks ago

    india, IN AL Solutions Full time

    AL Solutions are partnered with a mid-sized CRO who are searching for a freelance Biostatistician to work on an FSP basis and support their Oncology focused client.This is a full time, fully remote contract that will run for 12 months with possibility of extension.Design statistical analysis plans (SAPs) for oncology clinical trials and observational...

  • Freelance Flutter Developer

    3 weeks ago

    india, IN Leading MNC Part time

    Looking for a Freelance Flutter Developer to join a team of rockstar developers. The candidate should have a minimum of 5+ yrs. of experience .There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You will report...

  • Freelance Oracle CX Developer

    3 weeks ago

    india, IN Leading MNC Part time

    Looking for a Freelance Oracle CX Developer to join a team of rockstar developers. The candidate should have a minimum of 5+ yrs. of experience.There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You will report...

  • Data Engineer

    3 weeks ago

    , India, IN KPG99 INC Full time

    Role- Databricks EngineerLocation- Remote Duration- 12+ months with ExtensionsREQUIRED SKILLS AND EXPERIENCE- 3–5 years of experience in data engineering roles- Strong hands-on experience with Databricks for data processing and pipeline development.- Proficiency in SQL for data querying, transformation, and troubleshooting.- Solid programming skills in...

  • Data Engineer

    3 weeks ago

    india, IN Insight Global Full time

    Job DescriptionInsight Global is looking for a skilled senior level Data Engineer in Hyderabad or remote India for a large AAA gaming company on a contract for 3 months with possibility of extension or conversion. You will be assisting the Data and Analytics team on one of the company's largest game titles set to launch this fall. Your day-to-day will be...

  • Data Engineer

    3 weeks ago

    india, IN Insight Global Full time

    Position: GCP Data Engineer Location: 100% Remote in IndiaDuration: 12 month contract + extensions + conversionsPackage: 10 LPA- 26 LPAInterview Process: 2 RoundsREQUIRED SKILLS AND EXPERIENCE6+ Years of experience as a Data Engineer Experience with GCP Data ie. Big Query, Cloud Storage, BigTable, Airflow, Dataproc, Dataflow Strong SQL experience (NoSQL,...