Freelance Data Engineer

2 weeks ago

Thiruvananthapuram, India Whatjobs IN C2 Full time

We are seeking a highly skilled and motivated Data Engineer to join our team. The ideal candidate will be responsible for designing, developing, and optimizing large-scale data pipelines and data warehouse solutions, utilizing a modern, cloud-native data stack. You'll play a crucial role in transforming raw data into actionable insights, ensuring data quality, and maintaining the infrastructure required for seamless data flow. Key Responsibilities Develop, construct, test, and maintain robust and scalable large-scale ETL pipelines using PySpark for processing and Apache Airflow for workflow orchestration. Design and implement both Batch ETL and Streaming ETL processes to handle various data ingestion requirements. Build and optimize data structures and schemas in cloud data warehouses like AWS Redshift . Work extensively with AWS data services, including AWS EMR for big data processing, AWS Glue for serverless ETL, and Amazon S3 for data storage. Implement and manage real-time data ingestion pipelines using technologies like Kafka and Debezium for Change Data Capture (CDC). Interact with and integrate data from various relational and NoSQL databases such as MySQL , PgSQL (PostgreSQL) , and MongoDB . Monitor, troubleshoot, and optimize data pipeline performance and reliability. Collaborate with data scientists, analysts, and other engineering teams to understand data needs and deliver high-quality, reliable data solutions. Ensure data governance, security, and quality across all data platforms. Required Skills & Qualifications Technical Skills Expert proficiency in developing ETL/ELT solutions using PySpark . Strong experience in workflow management and scheduling tools, specifically Apache Airflow . In-depth knowledge of AWS data services including: AWS EMR (Elastic MapReduce) AWS Glue AWS Redshift Amazon S3 Proven experience implementing and managing data streams using Kafka . Familiarity with Change Data Capture (CDC) tools like Debezium . Hands-on experience with diverse database technologies: MySQL , PgSQL , and MongoDB . Solid understanding of data warehousing concepts, dimensional modeling, and best practices for both batch and real-time data processing. Proficiency in a scripting language, preferably Python . General Qualifications Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field. Excellent problem-solving, analytical, and communication skills. Ability to work independently and collaboratively in a fast-paced, dynamic environment. Nice to Have (Preferred Skills) Experience with Infrastructure as Code (e.G., Terraform, CloudFormation). Knowledge of containerization technologies (Docker, Kubernetes). Familiarity with CI/CD pipelines.

  • Freelance Data Science Trainer

    2 weeks ago

    Thiruvananthapuram, India INTTRVU.AI Full time

    About Us:INTTRVU Learning Platform LLP is an EdTech firm specializing in training individuals for roles in data science, artificial intelligence, and data analytics. Our mission is to empower aspiring professionals with the skills and knowledge needed to excel in the dynamic data field. We are experiencing robust growth and seeking outstanding individuals to...

  • Freelance Data Quality Automation

    2 weeks ago

    Thiruvananthapuram, India Leading MNC Full time

    Looking for a Freelance Data Quality Automation to join a team of rockstar developers. The candidate should have a minimum of 4+ yrs. of experience.There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You will...

  • Big Data Engineer

    1 week ago

    Thiruvananthapuram, Kerala, India Futuremug Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Big Data Engineer (Hadoop/Devops) Freelance InterviewerExperience -8 to 12 yearsAbout the companyWe are an HR Tech company based in Trivandrum, offering hiring support to MNCs across India through interview ,assessments and recruitment services. We have a network of 4000+ experienced professionals who take interviews in their available time slots.Were...

  • 2d designer freelance

    7 days ago

    Thiruvananthapuram, Kerala, India BRICK TREE INTERIORS (INTERIOR FIT OUT PROJECTS) Full time ₹ 2,40,000 - ₹ 7,20,000 per year

    Job Title: 2D Designer (Freelance) - InteriorsProject Overview:We're seeking a skilled 2D Designer with expertise in interior design to create detailed and visually appealing designs for residential and commercial spaces.Responsibilities:Create 2D designs, floor plans, and elevations for interior spacesDevelop design concepts, layouts, and visualizationsWork...

  • ML Engineer(Freelance Interviewer)

    1 week ago

    Thiruvananthapuram, Kerala, India Futuremug Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    ML Engineer(Freelance Interviewer)Experience:10 to 30 yearsLocation: RemoteExperience in developing and deploying Machine Learning models, Generative AI and Large Language Models (LLMs). The role involves developing, fine-tuning, and deploying ML and AI models for various applications. Key Responsibilities: Develop and optimize ML models Implement ML...

  • IVR Scripting-Freelance Interviewer

    2 weeks ago

    Thiruvananthapuram, Kerala, India Futuremug Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    IVR Scripting-Freelance InterviewerExperience:7 to 10yearsAbout the companyWe are an HR Tech company based in Trivandrum, offering hiring support to MNCs across India through interview ,assessments and recruitment services. We have a network of 4000+ experienced professionals who take interviews in their available time slots.Were looking for experienced...

  • 2d Designer Freelance

    7 days ago

    Thiruvananthapuram, Kerala, India BRICK TREE INTERIORS (INTERIOR FIT OUT PROJECTS) Full time

    **Job Title: 2D Designer (Freelance) - Interiors** **Project Overview**: We're seeking a skilled 2D Designer with expertise in interior design to create detailed and visually appealing designs for residential and commercial spaces. **Responsibilities**: - Create 2D designs, floor plans, and elevations for interior spaces - Develop design concepts,...

  • OpenShift ( Freelance Interviewer)

    1 week ago

    Thiruvananthapuram, Kerala, India Futuremug Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    OpenShift ( Freelance Interviewer)Experience - 5 to 10 yearsAbout the companyWe are an HR Tech company based in Trivandrum, offering hiring support to MNCs across India through interview ,assessments and recruitment services. We have a network of 4000+ experienced professionals who take interviews in their available time slots.Were looking for experienced...

  • Freelance Interviewer-NCino

    2 weeks ago

    Thiruvananthapuram, Kerala, India Futuremug Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Freelance Interviewer-NCinoExperience:7 to 10 yearsAbout the companyWe are an HR Tech company based in Trivandrum, offering hiring support to MNCs across India through interview ,assessments and recruitment services. We have a network of 4000+ experienced professionals who take interviews in their available time slots.Were looking for experienced...

  • Freelance Interviewer-Full Stack

    2 weeks ago

    Thiruvananthapuram, Kerala, India Futuremug Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Senior Full Stack (Optimizely)-Freelance InterviewerAbout the companyWe are an HR Tech company based in Trivandrum, offering hiring support to MNCs across India through interview ,assessments and recruitment services. We have a network of 4000+ experienced professionals who take interviews in their available time slots.Were looking for experienced...