Azure Data Engineer

1 week ago

Mumbai, Maharashtra, India Celebal Technologies Full time US$ 1,00,000 - US$ 1,50,000 per year

Overview: As a Data Engineer you will work with multiple teams to deliver solutions on the Azure Cloud using core cloud data warehouse tools (Azure Data Factory, Azure Databricks, Azure Synapse Analytics and other Big Data related technologies). In addition to building the next generation of application data platforms (not infrastructure) and/or improving recent implementations. Experience in Databricks and Spark on other cloud platforms like AWS and GCP is also relevant.

Responsibilities:


• Defines, designs, develops and test software components/applications using Microsoft Azure- (Azure Databricks, Azure Data Factory, Azure Data Lake Storage, Logic Apps, Azure SQL database, Azure Key Vaults, ADLS)


• Strong SQL skills with experience.


• Experience handling Structured and unstructured datasets.


• Experience in Data Modeling and Advanced SQL techniques


• Experience implementing Azure Data Factory, Airflow, AWS Glue or any other data orchestration tool using latest technologies and techniques.


• Good exposure in Application Development.


• The candidate should work independently with minimal supervision.

Must Have:


• Hands on experience with distributed computing framework like DataBricks, Spark- Ecosystem (Spark Core, PySpark, Spark Streaming, SparkSQL)


• Willing to work with product teamsto best optimize product features/functions.


• Experience on Batch workloads and real time streaming with high volume data frequency


• Performance optimization on Spark workloads


• Environment setup, user management, Authentication and cluster management on Databricks


• Professional curiosity and the ability to enable yourself in new technologies and tasks.


• Good understanding of SQL and a good grasp of relational and analytical database management theory and practice.

Good To Have:


• Hands on experience with distributed computing framework like DataBricks.


• Experience with Databricks migration from On-premise to Cloud OR Cloud to Cloud


• Migration of ETL workloadsfrom Apache Spark implementationsto Databricks


• Experience on Databricks ML will be a plus


• Migration from Spark 2.0 to Spark 3.5


• Certifications : Databricks Solution Architect Essentials badge Databricks Developer Essentials Apache Spark Programming with Databricks Data Engineering with Databricks Lakehouse with Delta Lake Deep Dive Fundamentals of Unified Data Analytics with Databricks

Key Skills:


• Python, SQL and Pyspark


• Big Data Ecosystem (Hadoop, Hive, Sqoop, HDFS, Hbase)


• Spark Ecosystem (Spark Core, Spark Streaming, Spark SQL) / Databricks


• Azure (ADF, ADB, Logic Apps, Azure SQL database, Azure Key Vaults, ADLS, Synapse)


• AWS (Lambda,AWS Glue, S3, Redshift)


• Data Modelling, ETL Methodology

  • Azure Data Engineer

    1 week ago

    Mumbai, Maharashtra, India LTIMindtree Full time

    Skill: Azure Data EngineerPrimary skill: ADB,ADF,SQLSecondary skill: Py spark or PythonExperience: 5-12YRSLocation: Hyderabad, Pune, Mumbai, Kolkata, Nagpur, Bangalore,ChennaiNotice period: Imm to 30 DaysJD: Proactive Q2 Azure Data Factory ADFP3 Job Description Azure Databricks DeveloperLocation India ISTOverview Data Engineer having good experience on Azure...

  • Azure Data Engineer

    5 days ago

    Mumbai, Maharashtra, India LTIMindtree Full time

    Skill: Azure Data Engineer Primary skill: ADB,ADF,SQL Secondary skill: Py spark or Python Experience: 5-12YRS Location: Hyderabad, Pune, Mumbai, Kolkata, Nagpur, Bangalore,Chennai Notice period: Imm to 30 Days JD: Proactive Q2 Azure Data Factory ADFP3 Job Description Azure Databricks Developer Location India IST Overview Data Engineer having good experience...

  • Azure Data Engineer

    3 days ago

    Mumbai, Maharashtra, India Infogain Full time US$ 90,000 - US$ 1,20,000 per year

    Roles & ResponsibilitiesJob Description:ResponsibilitiesDevelop a detailed project plan outlining tasks, timelines, milestones, and dependencies.Solutions architecture design and implementationUnderstand the source and outline the ADF structure. Design and schedule packages using ADF.Foster collaboration and communication within the team to ensure smooth...

  • Azure Data Engineers

    4 days ago

    Mumbai, Maharashtra, India Creative Hr Solutions Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    Sr.Azure Data Engineer (8-12 yrs) Hyd/Blr/Pan India | UK Shift Azure Databricks Engineer (6-12 yrs) Skills:Azure, ADF, Databricks, Python, PySpark, Kafka, SQL, AKS, Delta Lake

  • Azure Data Architect

    3 days ago

    Mumbai, Maharashtra, India beBeeDataEngineering Full time ₹ 1,20,00,000 - ₹ 2,50,00,000

    Azure Data Engineering Roles">Azure data engineers are responsible for designing, implementing and maintaining large-scale data processing systems on the Azure platform.">Developing Scalable Data Pipelines]To develop scalable data pipelines, we look for individuals who can design, implement and maintain efficient and reliable data processing systems. This...

  • Azure Data Engineer

    3 weeks ago

    Navi Mumbai, Maharashtra, India Celebal Technologies Full time

    Job Title: Azure Data Engineer Duration: Fulltime Positions: Multiple Experience Required: 3-5 Years & 6-8 Years Job Summary: We are looking for a highly skilled Azure Data Engineer with a strong background in real-time and batch data ingestion and big data processing, particularly using Kafka and Databricks. The ideal candidate will have a deep...

  • Azure Data Engineer

    3 weeks ago

    Navi Mumbai, Maharashtra, India Celebal Technologies Full time

    Job Title: Azure Data Engineer Duration: Fulltime Positions: Multiple Experience Required: 3-5 Years & 6-8 Years Job Summary: We are looking for a highly skilled Azure Data Engineer with a strong background in real-time and batch data ingestion and big data processing, particularly using Kafka and Databricks. The ideal candidate will have a deep...

  • Azure Data Architect

    1 week ago

    Mumbai, Maharashtra, India Infogain Full time US$ 1,50,000 - US$ 2,00,000 per year

    Roles & ResponsibilitiesCore SkillsIn-depth knowledge of Azure data services such as Azure SQL Database, Azure Cosmos DB, and Azure Data Lake Storage.Experience in designing and implementing scalable data solutions.Proficiency in data modeling and database design.Understanding of data governance and security practices.Experience12-14 YearsSkillsPrimary...

  • Azure Data Engineer

    1 week ago

    Mumbai, Maharashtra, India Prointegrate World It Consulting Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Key Responsibilities:Design, build, and maintain robust data pipelines to curate and ingest data efficiently.Work with Azure cloud services to deploy and manage scalable data solutions.Implement and maintain CI/CD pipelines for seamless data workflows.Collaborate with cross-functional teams to ensure data integrity, security, and accessibility.Key Skills &...

  • Azure Data Engineer

    3 days ago

    Mumbai, Maharashtra, India Deloitte Full time

    Your potential, unleashed. India's impact on the global economy has increased at an exponential rate and Deloitte presents an opportunity to unleash and realise your potential amongst cutting edge leaders, and organizations shaping the future of the region, and indeed, the world beyond. At Deloitte, your whole self to work, every day. Combine that with our...