Expert Data Automation Engineer

2 days ago

India beBeeDataEngineer Full time US$ 1,04,000 - US$ 1,30,878
Job Title

We are seeking an expert in data engineering with a strong background in automation, deployment, and modern DevOps practices. The ideal candidate will have hands-on experience in designing, implementing, and managing large-scale data pipelines using Python, SQL, and Spark.

">Key Responsibilities
  • Design, implement, and manage CI/CD pipelines using GitHub Actions or equivalent tools.
  • Configure and maintain GitHub branch protection rules and workflows to ensure smooth code integration and deployment.
  • Build, optimize, and debug Spark jobs in Databricks for large-scale data processing.
  • Automate Databricks deployments using Databricks Asset Bundles (DABs) or similar frameworks.
  • Develop, test, and maintain robust data pipelines using Python and SQL.
  • Implement unit testing, validation, and quality checks for data workflows.
  • Manage multi-environment deployments (Dev, Stage, Prod) ensuring reliability and consistency.
  • Write and maintain configuration files (YAML, JSON) for workflows, pipelines, and automation.
  • Troubleshoot and resolve configuration or syntax issues in YAML and related files.
  • Keep critical Databricks notebooks running reliably to ensure business continuity.
  • Migrate legacy reports into modern, self-serve dashboards (Snowflake, Tableau, Power BI).
  • Partner with cross-functional teams to understand needs and deliver data models, insights, and reporting they can trust.
  • Put guardrails in place with data contracts, quality checks, and documentation to ensure long-term reliability.
Requirements

The ideal candidate should possess:

  • Strong skills in SQL, Databricks (Spark), and Python/R.
  • Strong knowledge of GitHub and CI/CD practices (branch protection, GitHub Actions workflows).
  • Experience in implementing unit tests and quality checks in data workflows.
  • Knowledge of Databricks Asset Bundles (DABs) or similar automation tools.
  • Familiarity with YAML configuration management, debugging, and error resolution.
  • Strong problem-solving and debugging skills in data engineering workflows.
  • Experience with Snowflake or another cloud data warehouse.
  • Hands-on experience with Tableau/Power BI dashboards and a good sense for usability.
What We Offer

A competitive package and a dynamic work environment that fosters innovation and growth.

  • Data Engineer

    4 weeks ago

    India ProLegion Full time

    Job DescriptionJob Title: Data Engineering & Analytics ExpertLocation: RemoteType: ContractAbout the Role:We&aposre seeking a skilled Data Engineering & Analytics Expert to lead the development and automation of high-performance data pipelines. Youll work across real-time streaming systems and synthetic data generation to power advanced analytics and AI...

  • Senior Data Engineer

    3 weeks ago

    India Aptus Data Labs Full time

    Job Title: Senior Data EngineerLocation: RemoteExperience: 4+Employment Type: Full-TimeNotice- Looking for immediate joiners or who can join within 15-20 days.About the RoleAptus Data Labs is looking for a talented and proactive Senior Data Engineer to help build the backbone of our enterprise data and AI initiatives. You'll work on modern data lake...

  • QA Automation Tester

    5 days ago

    India NTT DATA Full time

    **Req ID**: 331096 We are currently seeking a QA Automation Tester to join our team in Remote, Karnātaka (IN-KA), India (IN). Who we are: NTT DATA America's strive to hire exceptional, innovative and passionate individuals who want to grow with us. Launch by NTT DATA is the culmination of the company's strategy to acquire and integrate the skills,...

  • Senior Web Scraping and Data Automation Expert

    1 day ago

    India beBeeDataAutomation Full time ₹ 9,00,000 - ₹ 12,00,000

    Senior Web Scraping and Data Automation ExpertWe are seeking a high-level web scraping/data extraction specialist for a long-term freelance project that requires advanced technical skills, reliability, and creativity.This mission involves building a strategic data system that extracts and structures data from various online sources with dynamic content,...

  • Marketing Automation Expert

    4 weeks ago

    India Lotus Interworks Full time

    Job DescriptionJob Title: Marketing Automation ExpertLocation: RemoteEmployment Type: Full-TimeShift: 7pm IST - 4am ISTAbout the Role:We are seeking an MarketingAutomation Expert to design, implement, and manage workflows that increase efficiency, reduce manual tasks, and integrate AI-driven solutions across business operations. This role requires deep...

  • Urgent AI Workflow Automation Expert

    3 weeks ago

    India BeGig Full time

    Job DescriptionAI Workflow Automation ExpertAbout BeGigBeGig is the leading tech freelancing marketplace. We empower innovative, early-stage, non-tech founders to bring their visions to life by connecting them with top-tier freelance talent. By joining BeGig, youre not just taking on one roleyoure signing up for a platform that will continuously match you...

  • Senior Data Engineer

    4 weeks ago

    India Asenium Consulting Full time

    For one of my customers, we are looking for a Senior Data Engineer.Location: India (100% remote)Duration: 6+ months (renewable)Description: We are looking for a Senior Data Engineer with strong experience in SQL, Talend, and cloud platforms such as Google Cloud (Big Query) and Microsoft Azure.You will be responsible for designing and managing ETL pipelines,...

  • Quality Assurance Automation Engineer

    1 day ago

    India Datya Solutions Private Limited Full time

    Company DescriptionDatya Solutions Private Limited is headquartered in Davanagere, Karnataka, and is a bootstrapped company driven by a dedicated team of software developers and data analytics professionals. Our team is led by seasoned experts with 15-20 years of experience, committed to delivering exceptional results. We provide software development, data...

  • Automation Engineer

    19 hours ago

    India Bechtel Full time

    Requisition ID 277707 Relocation Authorized National International - Single Telework Type Full-Time Office Project Work Location Dahej Extraordinary teams building inspiring projects Since 1898 we have helped customers complete more than 25 000 projects in 160 countries on all seven continents that have created jobs grown economies improved the...

  • QA Automation Engineer

    2 days ago

    India Origins AI: Helping you build and scale Full time

    QA Automation Engineer (Playwright)About Us We are an IT services and AI solutions company helping global clients build reliable, scalable, and high-quality products. Our engineering culture blends deep technical expertise with AI-driven development and testing practices to ensure speed, precision, and unmatched quality.Role Overview We are seeking a QA...