Senior Azure Databricks Engineer

2 days ago

Hyderabad, Telangana, India Innova AM Tech Full time ₹ 12,00,000 - ₹ 24,00,000 per year

About the Role

  • We are seeking a highly skilled and experienced Senior Azure Databricks Engineer to join our dynamic data engineering team.

  • As a Senior Azure Databricks Engineer, you will play a critical role in designing, developing, and implementing data solutions on the Azure Databricks platform.

  • You will be responsible for building and maintaining high-performance data pipelines, transforming raw data into valuable insights, and ensuring data quality and reliability.

Key Responsibilities

  • Design, develop, and implement data pipelines and ETL/ELT processes using Azure Databricks.

  • Develop and optimize Spark applications using Scala or Python for data ingestion, transformation, and analysis.

  • Leverage Delta Lake for data versioning, ACID transactions, and data sharing.

  • Utilize Delta Live Tables for building robust and reliable data pipelines.

  • Design and implement data models for data warehousing and data lakes.

  • Optimize data structures and schemas for performance and query efficiency.

  • Ensure data quality and integrity throughout the data lifecycle.

  • Integrate Azure Databricks with other Azure services (e.g., Azure Data Factory, Azure Synapse Analytics, Azure Blob Storage).

  • Leverage cloud-based data services to enhance data processing and analysis capabilities.

Performance Optimization & Troubleshooting

  • Monitor and analyze data pipeline performance.

  • Identify and troubleshoot performance bottlenecks.

  • Optimize data processing jobs for speed and efficiency.

  • Collaborate effectively with data engineers, data scientists, data analysts, and other stakeholders.

  • Communicate technical information clearly and concisely.

  • Participate in code reviews and contribute to the improvement of development processes.

Qualifications

Essential

  • 5+ years of experience in data engineering, with at least 2 years of hands-on experience with Azure Databricks.

  • Strong proficiency in Python and SQL.

  • Expertise in Apache Spark and its core concepts (RDDs, DataFrames, Datasets).

  • In-depth knowledge of Delta Lake and its features (e.g., ACID transactions, time travel).

  • Experience with data warehousing concepts and ETL/ELT processes.

  • Strong analytical and problem-solving skills.

  • Excellent communication and interpersonal skills.

  • Bachelor's degree in Computer Science, Computer Engineering, or a related field.

  • Azure Databricks Engineer

    1 week ago

    Hyderabad, Telangana, India Tekskills Inc. Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Role: Azure Databricks EngineerJob Location: Pune, MH / Chennai, TN / Hyderabad. TG / Indore, MP / Kolkata, WB / Bangalore, TNWork Mode: (WFO)Skills Required: Digital: DatabricksExperience: 6-8 Years(5+ yrs)Job Description:Should build Databricks Notebook developed in spark sql/pysparkShould be able to create unity catalog tablesShould have good hands on...

  • Azure + Databricks SME

    1 week ago

    Hyderabad, Telangana, India Mindsstar Technology Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Experience:8-12 yearsSkills:Databricks, Unity Catalog, new data bricks offerings (i.e., Genie ), Azure , Data Management, Data EngineeringLocation - HyderabadResponsibilities:Architect, design and implement scalable, secure, and high-performance data platforms using Azure and Databricks.Go-to expert on Azure Data Services and Databricks platform...

  • Azure Databricks With terraform

    2 weeks ago

    Hyderabad, Telangana, India Adv Techminds Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    We are looking for Freelance who can support Daily 2-2.5 hours ,Below is the Requirment :relevant experiance must be : 7+ yrsHeavy on Azure databricks and terraformDevops position dealing with data and infrastructureDeploying Databricks cluster using terraformwe have to convert unity catalog to terraform in a monthAzure and azure DatabricksIf You have...

  • Senior Azure Data Engineer

    1 day ago

    Hyderabad, Telangana, India Ensar Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job SummaryWe are seeking a highly skilled Senior Azure Data Engineer with strong experience in integrating SAP systems with Azure cloud data platforms. The ideal candidate should be proficient in Azure Data Factory (ADF), Azure Databricks, and Azure Logic Apps, and have a deep understanding of building and maintaining robust, scalable, and secure data...

  • Senior Engineer – EIM – Azure/Databricks

    1 week ago

    Hyderabad, Telangana, India The TJX Companies, Inc. Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    TJX CompaniesAt TJX Companies, every day brings new opportunities for growth, exploration, and achievement. You'll be part of our vibrant team that embraces diversity, fosters collaboration, and prioritizes your development. Whether you're working in our four global Home Offices, Distribution Centers or Retail Stores—TJ Maxx, Marshalls, Homegoods,...

  • Databricks with Python

    2 weeks ago

    Hyderabad, Telangana, India Tata Consultancy Services Full time ₹ 1,00,00,000 - ₹ 2,00,00,000 per year

    Role:Databricks with PythonExperience range: 5-8 yearsLocation: HyderabadNOTE: Relevant Databricks experience must be 5 years and only apply if interested for WALK-IN Drive in HyderabadJob description:Must Have:Extensive expertise in designing and implementing data load processes using Azure Data Factory, Azure Databricks, Delta Lake, Azure Delta Lake...

  • Databricks with Python

    1 day ago

    Hyderabad, Telangana, India Tata Consultancy Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role - Databricks with PythonExp - 5 to 9 YrsLocation - HyderabadNote - This is a scheduled drive. Only eligible candidates who receive invite from us will be allowed inside Campus.Job DescriptionExtensive expertise in designing and implementing data load processes using Azure Data Factory, Azure Databricks, Delta Lake, Azure Delta Lake Storage and...

  • Senior Databricks Engineer

    1 day ago

    Hyderabad, Telangana, India Xenon7 Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    About us:Where elite tech talent meets world-class opportunitiesAt Xenon7, we work with leading enterprises and innovative startups on exciting, cutting-edge projects that leverage the latest technologies across various domains of IT including Data, Web, Infrastructure, AI, and many others. Our expertise in IT solutions development and on-demand resources...

  • Senior Databricks Engineer

    3 days ago

    Hyderabad, Telangana, India SID Information Technologies Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Role: Senior Data Engineer (Python, Spark/Databricks, SQL, AWS)Experience:6–12 yearsLocation:HyderabadWork Mode:Hybrid (3 days/week in-office)Join Time:ImmediateMust-Have Technical Skills:Strong programming skills in Python, DatabricksHands-on experience with Apache Spark & Databricks for big data processing on AWS cloudProficiency with AWS services ...

  • NET Full Stack with Azure Databricks

    2 weeks ago

    Hyderabad, Telangana, India DXC Technology Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    Job Description:C# or Java they are the same thing 2. Databricks 3. Azure 4. SQL 5. DevOps 6. .Net FrontEnd/Angular JS Responsible for leveraging cutting edge technology to solve business problems at American Airlines by participating in all phases of the development process from inception through transition, advocating the agile process and test-driven...