Sr. Data Engineer – AWS Databricks

3 weeks ago

Kochi, India Gravity Infosolutions, Inc. Full time

Sr. Data Engineer – AWS Databricks / Data Tech LeadType: ContractLocation : Chennai / HydExperience: 5+ yearsDuration: 3 Months+ Extendable Key Responsibilities Data Pipeline Development: Design and implement robust ETL/ELT pipelines using Databricks, PySpark, and Delta Lake to process structured and unstructured data efficiently.Manage job orchestration, scheduling, and workflow automation through Databricks Workflows or Airflow.Performance Optimization: Tune and optimize Databricks clusters and notebooks for performance, scalability, and cost-efficiency.Data Governance: Implement data governance and lineage using Unity Catalog and other platform-native featuresCollaboration: Work closely with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions that meet business needs.Cloud Integration: Leverage cloud platforms (AWS) to build and deploy data solutions, ensuring seamless integration with existing infrastructure.Data Modeling: Develop and maintain data models that support analytics and machine learning workflows.Automation & Monitoring: Implement automated testing, monitoring, and alerting mechanisms to ensure data pipeline reliability and data quality.Documentation & Best Practices: Maintain comprehensive documentation of data workflows and adhere to best practices in coding, version control, and data governance. Required Qualifications Experience: 5+ years in data engineering, with hands-on experience using Databricks and Apache Spark.Programming Skills: Proficiency in Python and SQL; Cloud Platforms: Strong experience with cloud services such as AWS (e.g., S3, Glue, RedshiftData Engineering Tools: Familiarity with tools like Airflow, Kafka, and dbt.Data Modeling: Experience in designing data models for analytics and machine learning applications.Collaboration: Proven ability to work in cross-functional teams and communicate effectively with non-technical stakeholders. Primary Skill Set Databricks, Apache Spark/Pyspark, Python, SQL, ETL/ELT development, Delta Lake, Cloud platforms (AWS), Data modeling, Cross-functional collaboration, Communication Secondary Skill SetAirflow, dbt, Kafka, Hadoop, MLflow, Unity Catalog, Delta Live Tables, Cluster optimization, Data governance, Security and compliance, Databricks certifications

  • Data Engineer

    3 weeks ago

    Kochi, India Exult Global Full time

    Role: Data Engineer Experience: 3-5 yrsNotice Period: ImmediateWork Mode: HybridJob DescriptionRequirements:• Proficient in Python (Including popular python packages e.g. Pandas, NumPy etc.) and SQL• Strong background in distributed data processing and storage (e.g. Apache, Spark, Hadoop)• Large scale (TBs of data) data engineering skills - Model data,...

  • Data Engineer

    3 weeks ago

    Kochi, India Exult Global Full time

    Role: Data Engineer Experience: 3-5 yrs Notice Period: Immediate Work Mode: Hybrid Job Description Requirements: • Proficient in Python (Including popular python packages e.g. Pandas, NumPy etc.) and SQL • Strong background in distributed data processing and storage (e.g. Apache, Spark, Hadoop) • Large scale (TBs of data) data engineering skills -...

  • Data Engineer

    7 days ago

    Kochi, India Exult Global Full time

    Role: Data Engineer Experience: 3-5 yrs Notice Period: Immediate Work Mode: Hybrid Job Description Requirements: • Proficient in Python (Including popular python packages e.g. Pandas, NumPy etc.) and SQL • Strong background in distributed data processing and storage (e.g. Apache, Spark, Hadoop) • Large scale (TBs of data) data engineering skills -...

  • Data Engineer

    3 weeks ago

    Kochi, India Exult Global Full time

    Role: Data Engineer Experience: 3-5 yrs Notice Period: Immediate Work Mode: Hybrid Job Description Requirements: • Proficient in Python (Including popular python packages e.g. Pandas, NumPy etc.) and SQL • Strong background in distributed data processing and storage (e.g. Apache, Spark, Hadoop) • Large scale (TBs of data) data engineering skills -...

  • Data Engineer

    2 weeks ago

    Kochi, India Exult Global Full time

    Role: Data EngineerExperience: 3-5 yrsNotice Period: ImmediateWork Mode: HybridJob DescriptionRequirements:• Proficient in Python (Including popular python packages e.g. Pandas, NumPy etc.) and SQL• Strong background in distributed data processing and storage (e.g. Apache, Spark, Hadoop)• Large scale (TBs of data) data engineering skills - Model data,...

  • Data Engineer

    1 week ago

    Kochi, India Exult Global Full time

    Role: Data EngineerExperience: 3-5 yrsNotice Period: ImmediateWork Mode: HybridJob DescriptionRequirements:• Proficient in Python (Including popular python packages e.g. Pandas, NumPy etc.) and SQL• Strong background in distributed data processing and storage (e.g. Apache, Spark, Hadoop)• Large scale (TBs of data) data engineering skills - Model data,...

  • Data Engineer

    3 weeks ago

    Kochi, India Exult Global Full time

    Role: Data EngineerExperience: 3-5 yrsNotice Period: ImmediateWork Mode: Hybrid Job DescriptionRequirements:• Proficient in Python (Including popular python packages e.g. Pandas, NumPy etc.) and SQL• Strong background in distributed data processing and storage (e.g. Apache, Spark, Hadoop)• Large scale (TBs of data) data engineering skills - Model data,...

  • Data engineer

    2 weeks ago

    Kochi, India Exult Global Full time

    Role: Data EngineerExperience: 3-5 yrsNotice Period: ImmediateWork Mode: Hybrid Job DescriptionRequirements:• Proficient in Python (Including popular python packages e.g. Pandas, Num Py etc.) and SQL• Strong background in distributed data processing and storage (e.g. Apache, Spark, Hadoop)• Large scale (TBs of data) data engineering skills - Model...

  • Senior Data Engineer

    1 day ago

    kochi, India Milestone Technologies, Inc. Full time

    Role Overview:Seeking an experienced Senior Data Engineer to lead data ingestion, transformation, and optimization initiatives using the modern Apache and Azure data stack. The role involves working on scalable pipelines, large-scale distributed systems, and data lake management.Core Responsibilities:· Build and manage high-volume data pipelines using...

  • Lead data engineer

    3 weeks ago

    Kochi, India Cozzera Full time

    Job Title: Lead Data EngineerExperience: 8–10 YearsLocation: Remote ( 1 week from noida office then remote)Job Overview:We are looking for an accomplished Lead Data Engineer with 8–10 years of experience inarchitecting, building, and managing complex data ecosystems using Python, Databricks(Delta Lake, Spark, MLflow), and modern orchestration frameworks....