Freelance data engineer

4 days ago

Davangere, India UpGrad Full time

We are seeking a highly skilled and motivated Data Engineer to join our team. The ideal candidate will be responsible for designing, developing, and optimizing large-scale data pipelines and data warehouse solutions, utilizing a modern, cloud-native data stack. You'll play a crucial role in transforming raw data into actionable insights, ensuring data quality, and maintaining the infrastructure required for seamless data flow.Key ResponsibilitiesDevelop, construct, test, and maintain robust and scalable large-scale ETL pipelines using Py Spark for processing and Apache Airflow for workflow orchestration.Design and implement both Batch ETL and Streaming ETL processes to handle various data ingestion requirements.Build and optimize data structures and schemas in cloud data warehouses like AWS Redshift.Work extensively with AWS data services, including AWS EMR for big data processing, AWS Glue for serverless ETL, and Amazon S3 for data storage.Implement and manage real-time data ingestion pipelines using technologies like Kafka and Debezium for Change Data Capture (CDC).Interact with and integrate data from various relational and No SQL databases such as My SQL, Pg SQL (Postgre SQL), and Mongo DB.Monitor, troubleshoot, and optimize data pipeline performance and reliability.Collaborate with data scientists, analysts, and other engineering teams to understand data needs and deliver high-quality, reliable data solutions.Ensure data governance, security, and quality across all data platforms.Required Skills & QualificationsTechnical SkillsExpert proficiency in developing ETL/ELT solutions using Py Spark.Strong experience in workflow management and scheduling tools, specifically Apache Airflow.In-depth knowledge of AWS data services including:AWS EMR (Elastic Map Reduce)AWS GlueAWS RedshiftAmazon S3Proven experience implementing and managing data streams using Kafka.Familiarity with Change Data Capture (CDC) tools like Debezium.Hands-on experience with diverse database technologies: My SQL, Pg SQL, and Mongo DB.Solid understanding of data warehousing concepts, dimensional modeling, and best practices for both batch and real-time data processing.Proficiency in a scripting language, preferably Python.General QualificationsBachelor’s or Master’s degree in Computer Science, Engineering, or a related field.Excellent problem-solving, analytical, and communication skills.Ability to work independently and collaboratively in a fast-paced, dynamic environment.Nice to Have (Preferred Skills)Experience with Infrastructure as Code (e.g., Terraform, Cloud Formation).Knowledge of containerization technologies (Docker, Kubernetes).Familiarity with CI/CD pipelines.

  • Freelance data scientist

    1 week ago

    Davangere, India Leading MNC Full time

    Looking for a Freelance Data Scientist to join a team of rockstar developers. The candidate should have a minimum of 4+ yrs. of experience. There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You will report into...

  • Freelance Collibra Developer

    3 weeks ago

    Davangere, India Leading MNC Full time

    Looking for a Freelance Collibra Developer to join a team of rockstar developers. The candidate should have a minimum of 4+ yrs. of experience.There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You will report...

  • Lead Backend Engineer

    3 weeks ago

    Davangere, India Deccan AI Full time

    About Deccan AI AtDeccan AI , we’re reimagining the future of work by building afreelancer platformpowered by AI. Fromonboarding and assessmenttodeployment and payments , we aim to deliver a seamless, trusted, and scalable freelancer experience. If you’re passionate aboutdistributed systems, AI-native workflows, and user-first design , this is your...

  • Data Engineer

    2 days ago

    Davangere, India Collabera Full time

    Data EngineerREMOTE, INDIADay to day:As a Data Engineer, you will design, build, and maintain data products such as pipelines, models, APIs, and visualizations using best-in-class tools and DataOps practices. Your role involves creating scalable solutions that integrate machine learning and AI capabilities, implementing software engineering standards like...

  • Senior data engineer

    1 week ago

    Davangere, India Atomic North Full time

    Position: Senior Data Engineer (Azure) Experience: 6+ years Skills: Azure Databricks, Py Spark, Python, SQL, Azure Data Factory, Data Modelling, Data Migration, Data Warehousing Location: India (Remote / Hybrid options available) Interview Mode: Virtual Notice Period: Immediate to 30 Days ???? Key Responsibilities Design and develop data pipelines and ETL...

  • Développeur web

    3 weeks ago

    Davangere, India 360 Space Full time

    LONG-TERM FREELANCE - Expert Web Scraping/Data ExtractionWe are looking for a high-level scraper / data automation expert ready to take on a long-term project with serious ambition. We’re talking about a smart, evolving system that requires advanced technical skill, reliability, and creativity to bypass complex data access challenges.💡 About the mission...

  • Data/ML Engineer

    4 weeks ago

    Davangere, India Lilo Full time

    Lilo is revolutionizing procurement for Commercial Real Estate (CRE) businesses by providing a single platform to automate and optimize procurement processes using AI. From hotels to gyms, schools, and senior living homes, we streamline workflows like invoicing, vendor management, and price comparisons, saving our clients both time and money. We are already...

  • Technical lead

    1 week ago

    Davangere, India Aceolution Full time

    Freelance Remote Opportunity: Tech Lead – Gen AI Code Initiatives We’re seeking an experienced Tech Lead / Senior Software Engineer to spearhead our Gen AI Code Initiatives . This is a hands-on, freelancing remote role focused on evaluating, improving, and advancing AI-driven code generation systems. Key Responsibilities Code Generation & Refinement:...

  • Backend engineer — data

    3 weeks ago

    Davangere, India Branch International Full time

    Branch OverviewImagine a world where every person has improved access to financial services. People could start new businesses, pay for their children’s education, cover their emergency medical bills – the possibilities to improve life are endless.Branch is a global technology company revolutionizing financial access for millions of underserved banking...

  • Freelance Digital Strategist

    3 weeks ago

    Davangere, India nicheBrains Full time

    Industry: Textile Manufacturing Location: Hyderabad Employment Type: Contract (Remote)Company Description: nicheBrains is a modern digital transformation platform focused on empowering Micro, Small, and Medium Enterprises (MSMEs). Our platform connects business leaders from the textile industry with vetted experts to accelerate growth, innovation, and...