Data Engineering Leader

2 weeks ago

Mumbai Pune, India beBeeDataEngineering Full time US$ 90,000 - US$ 1,20,000
Job Title

We are seeking an expert Data Engineering leader to manage and maintain our Databricks infrastructure. The successful candidate will have extensive experience in managing large-scale distributed systems, implementing automation scripts using Python and Spark, and providing exceptional support to our analyst community.

About the Role
  • Key Responsibilities:
  • Manage and optimize Databricks clusters at scale, ensuring high availability and performance.
  • Oversee deployments within Databricks and work with Databricks System tables for monitoring and management.
  • Provide strong troubleshooting capabilities for issues within Databricks notebooks and address Databricks-related queries from the analyst community.
  • Apply good knowledge of AWS services and strong fundamentals of Data Warehousing (DW) / Data Lakehouse (DH) concepts.
  • Utilize Spark/Python for scripting and automation within the Databricks environment.
  • Work with Docker and Kubernetes.
  • Leverage knowledge of relevant AWS services such as S3, Lambda, and EMR.
Requirements
  • Good knowledge of AWS, Databricks, and Unity Catalog.
  • Strong experience in managing Databricks clusters and deployments.
  • Strong fundamentals of Data Warehousing (DW) / Data Lakehouse (DH) concepts.
  • Strong troubleshooting abilities within the Databricks environment.
  • Proficiency in Spark/Python (either coding skills).
  • Experience with Docker, Kubernetes, and AWS services (S3, Lambda, EMR).
  • Expertise in managing clusters at scale and troubleshooting within Databricks.
  • Experience working with Databricks System tables.
  • Strong knowledge of Databricks infrastructure and administration to support the Analyst community in addressing Databricks-related queries and troubleshooting issues in notebooks.
Qualifications
  • Bachelor's degree in Computer Science, Engineering, or a related field.
  • Proven experience as a Databricks Admin or in a similar role.
  • Typically seeking candidates with 6+ years of relevant experience.
  • Certified Databricks Engineer is required.

  • Data Architect

    1 week ago

    Pune, India Change Leader Full time

    From 11 to 17 year(s) of experience - ₹ Not Disclosed by Recruiter - Pune**Roles and Responsibilities** An experienced hands-on data architect with in-depth, hands-on knowledge of foundational data architectures such as data warehouses, ETLs and in-memory OLAP models, as well as experience in NoSQL and cloud implementations. In this role, employee will...

  • Data Engineer

    2 weeks ago

    Pune, Maharashtra, India Data Axle Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    About Data Axle:Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and leveraging proprietary...

  • Data Engineer

    4 days ago

    Pune, India Data Axle Full time

    About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and leveraging...

  • Azure Data Engineer

    2 weeks ago

    Pune, Maharashtra, India Capgemini Engineering Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    At Capgemini Engineering, the world leader in engineering services, we bring together a global team of engineers, scientists, and architects to help the world's most innovative companies unleash their potential. From autonomous cars to life-saving robots, our digital and software technology experts think outside the box as they provide unique R&D and...

  • Azure Data Engineer

    4 days ago

    Pune, India Capgemini Engineering Full time

    At Capgemini Engineering, the world leader in engineering services, we bring together a global team of engineers, scientists, and architects to help the world's most innovative companies unleash their potential. From autonomous cars to life-saving robots, our digital and software technology experts think outside the box as they provide unique R&D and...

  • Cloud and Data Engineering Leader

    1 week ago

    Pune, Maharashtra, India beBeeEngineering Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Cloud and Data Engineering Leader">We are seeking a highly skilled and motivated engineering leader to join our team as a Cloud and Data Engineer. The ideal candidate will possess extensive expertise in cloud technologies and data engineering, with a proven ability to orchestrate cutting-edge technology to connect various applications within the cloud...

  • Data Modeler with Snowflake

    1 day ago

    Pune, India Change Leader Full time

    **JOB DESCRIPTION** **Data Modeler with Snowflake** **Experience**: 8 - 12 years **Locations**:Pune, Chennai, Bangalore, Mumbai **Primary Skills**:Data Modeler's experience, Relational data modeling, Hierarchical data modeling & No-SQL modeling. Experience in Snowflake data modeling, Erwin / Hackolade **Desired Skills & Responsibilities** - **8+ years...

  • Data Architect

    1 week ago

    Mumbai, India Change Leader Full time

    ₹ Not Disclosed by Recruiter - Mumbai, Pune, Chennai**Roles and Responsibilities** Experience in Data Space across Architecture, Design, Development, testing & Deployment, full understanding in SDLC. Required to be hands on Tech Architectural profile, providing solutions, suggestions and helping in maintaining the technical frameworks. - Expert level...

  • Data Engineer

    4 days ago

    Pune, India Jash Data Sciences Full time

    Do you love solving real-world data problems with the latest and best techniques? And having fun while solving them in a team Then come join our high-energy team of passionate data people. Jash Data Sciences is the right place for you. We are a cutting-edge Data Sciences and Data Engineering startup based in Pune, India. We believe in continuous learning and...

  • Aws & Pyspark

    1 week ago

    Mumbai, India Change Leader Full time

    From 4 to 9 year(s) of experience - ₹ Not Disclosed by Recruiter - Mumbai, Pune, Chennai, Bangalore/Bengaluru**Roles and Responsibilities** **Must haveskills**: 1.PySpark(Python +Spark) or Spark with Scala. Spark aka Apache Spark. 2.Cloud(AWSorAzure). Preferably AWS Good to have:Databricks(PySpark),Snowflake.AWS(S3, Glue,...) **Primary skill**: Data...