Sr. Data Engineer – AWS Databricks

2 weeks ago

Hyderabad, Telangana, India Gravity Infosolutions, Inc. Full time ₹ 8,00,000 - ₹ 12,00,000 per year

Sr. Data Engineer – AWS Databricks / Data Tech Lead

Type: Contract

Location : Chennai / Hyd

Experience: 5+ years

Duration: 3 Months+ Extendable

Key Responsibilities

  • Data Pipeline Development:
    Design and implement robust ETL/ELT pipelines using Databricks, PySpark, and Delta Lake to process structured and unstructured data efficiently.
  • Manage job orchestration, scheduling, and workflow automation through
    Databricks Workflows
    or
    Airflow
    .
  • Performance Optimization:
    Tune and optimize Databricks clusters and notebooks for performance, scalability, and cost-efficiency.
  • Data Governance:
    Implement
    data governance and lineage
    using
    Unity Catalog
    and other platform-native features
  • Collaboration:
    Work closely with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions that meet business needs.
  • Cloud Integration:
    Leverage cloud platforms (AWS) to build and deploy data solutions, ensuring seamless integration with existing infrastructure.
  • Data Modeling:
    Develop and maintain data models that support analytics and machine learning workflows.
  • Automation & Monitoring:
    Implement automated testing, monitoring, and alerting mechanisms to ensure data pipeline reliability and data quality.
  • Documentation & Best Practices:
    Maintain comprehensive documentation of data workflows and adhere to best practices in coding, version control, and data governance.

Required Qualifications

  • Experience:
    5+ years in data engineering, with hands-on experience using Databricks and Apache Spark.
  • Programming Skills:
    Proficiency in Python and SQL;
  • Cloud Platforms:
    Strong experience with cloud services such as AWS (e.g., S3, Glue, Redshift
  • Data Engineering Tools:
    Familiarity with tools like Airflow, Kafka, and dbt.
  • Data Modeling:
    Experience in designing data models for analytics and machine learning applications.
  • Collaboration:
    Proven ability to work in cross-functional teams and communicate effectively with non-technical stakeholders.

Primary Skill Set

Databricks, Apache Spark/Pyspark, Python, SQL, ETL/ELT development, Delta Lake, Cloud platforms (AWS), Data modeling, Cross-functional collaboration, Communication

Secondary Skill Set

Airflow, dbt, Kafka, Hadoop, MLflow, Unity Catalog, Delta Live Tables, Cluster optimization, Data governance, Security and compliance, Databricks certifications

  • AWS-Databricks Engineer

    2 weeks ago

    Hyderabad, Telangana, India Coforge Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    We are seeking an experiencedAWS Databricks Developerto design, develop, and maintain scalable data processing and analytics solutions on theAWS and Databricksplatform. The ideal candidate will have strong expertise in building data pipelines, optimizing performance, and integrating multiple AWS services to deliver high-quality data solutions.Key...

  • AWS Databricks

    2 days ago

    Hyderabad, Telangana, India LTIMindtree Full time

    Job Location-ALL LTI LocationExp-5 to 12 yearsDatabricks expertise Proven HandsOn experience with the Databricks platform and its componentsProgramming languages Strong proficiency in PySpark for data manipulation and processingBig data technologies Expertise in Apache Spark and Delta Lake for handling large datasetsCloud knowledge Good to have Experience...

  • Data Engineering Manager

    3 weeks ago

    Hyderabad, Telangana, India, Telangana Blend Full time

    We are looking for an experienced Data Engineering Manager with a strong foundation in Python, SQL, and Spark, and hands-on expertise in AWS, Databricks. In this role, you will build and maintain scalable data pipelines and architecture to support analytics, data science, and business intelligence initiatives. You’ll work closely with cross-functional...

  • Databricks Engineer

    6 days ago

    Hyderabad, Telangana, India NTT DATA Full time

    Design, build, and maintain robust ETL/ELT data pipelines using Apache Spark on Databricks. Implement data Lakehouse architecture using Delta Lake for cost-effective data storage and analytics. Use Databricks Workflows for orchestrating batch and streaming pipelines. Develop and maintain CI/CD pipelines for data applications using tools such as Azure DevOps,...

  • Databricks with AWS

    2 days ago

    Hyderabad, Telangana, India Cognizant Technology Solutions Full time

    AIA - Hyderabed DriveJob SummaryWe are seeking a Sr. Developer with 7 to 12 years of experience to join our team. The ideal candidate will have expertise in Spark in Scala Delta Sharing and Databricks Unity Catalog Admin. This role involves working with cutting-edge technologies like Databricks CLI Delta Live Pipelines and Structured Streaming. The candidate...

  • AWS Data Engineer

    2 days ago

    Hyderabad, Telangana, India Virtusa Full time

    AWS Data EngineerWorkMode \:HybridWork Location \: Chennai / Hyderabad / Bangalore / Pune /Gurgaon / MumbaiWork Timing \: 2 PM to 11 PMPrimary \: AWS Data EngineerAWS Glue, Redshift, Python, ETL.Design and build scalable data pipelines using AWS services like AWS Glue, Amazon Redshift, and S3.Develop efficient ETL processes for data extraction,...

  • Databricks with AWS Admin

    2 days ago

    Hyderabad, Telangana, India Tech Mahindra Full time

    Role & responsibilities5-8 years of experience in Databricks and AWS administration and cloud infrastructure managementStrong knowledge of Databricks architecture, clusters, jobs and Unity CatalogStrong knowledge of AWS core services (Glue, Redshift, S3, EC2 etc)Strong grasp of network security concepts, VPC configuration, firewall rules and private link...

  • aws databricks with devops

    4 days ago

    Hyderabad, Telangana, India Primus Global Technologies Full time

    Type of RoleClient-facing roleMust to Have SkillSenior: 6+ YOE overall in DevOps2+ hands on databricksDevOps LeadDriving automationCI/CD pipelines optimising deploymentsInfra + code deploymentsCollab with data engineers across teamsAutomation process and DevOps solutionsGitHub and Databricks DevOps, AWSAzure DevOps/Integrated with GitHubControl

  • Databricks Engineer

    2 weeks ago

    Hyderabad, Telangana, India, Telangana iDWTeam Full time

    We are seeking an experienced Databricks Engineer to join our data team. The ideal candidate will have strong expertise in Databricks, Spark, and cloud platforms (Azure/AWS/GCP), along with a solid understanding of data engineering best practices. You will work closely with cross-functional teams to build scalable data pipelines, optimize big-data workloads,...

  • Senior Data Engineer – Databricks

    2 days ago

    Hyderabad, Telangana, India S2Integrators Full time

    Job Title:Senior Data Engineer – DatabricksExperience:6–10 YearsWork Mode:HybridShift Timing:4:00 PM – 1:00 AM ISTRole OverviewWe are looking for an experiencedSenior Data Engineerwith strong hands-on exposure toDatabricksto design, build, and optimize scalable data pipelines and analytics platforms. The role involves working closely with...