Azure Data Engineer-Technical Lead

3 weeks ago

Kolkata, India Sigmoid Full time

Kindly find the Job Description Below. Job Title:Technical Lead-Azure Data Engineer Location: Bangalore Years of Experience: 10+ years of experience Sigmoid works with a variety of clients from start-ups to fortune 500 companies. We are looking for a detailed oriented self-starter to assist our engineering and analytics teams in various roles as a Software Development Engineer. As an Technical Lead- Data Engineer, you will be responsible for building a highly-scalable and extensible big data platform that provides the foundation for collecting, storing, modeling, and analyzing massive data sets from multiple channels. We are looking for a skilled Data Engineer with 10+ years of experience in big data technologies, particularly Azure, Python, PYSpark, SQL and data lakehouse architectures. The ideal candidate will have a strong background in building scalable data pipelines and experience with modern data storage formats, including Apache Iceberg. You will work closely with cross-functional teams to design and implement efficient data solutions in a cloud-based environment. Key Responsibilities: Data Pipeline Development: Design, build, and optimize scalable data pipelines using Apache Spark. Implement and manage large-scale data processing solutions across data lakehouses. Data Lakehouse Management: Work with modern data lakehouse platforms (e.g.Apache Iceberg) to handle large datasets. Optimize data storage, partitioning, and versioning to ensure efficient access and querying. SQL & Data Management: Write complex SQL queries to extract, manipulate, and transform data. Develop performance-optimized queries for analytical and reporting purposes. Data Integration: Integrate various structured and unstructured data sources into the lakehouse environment. Work with stakeholders to define data needs and ensure data is available for downstream consumption. Data Governance and Quality: Implement data quality checks and ensure the reliability and accuracy of data. Contribute to metadata management and data cataloging efforts. Performance Tuning: Monitor and optimize the performance of Spark jobs, SQL queries, and overall data infrastructure. Work with cloud infrastructure teams to optimize costs and scale as needed. Qualifications: Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field. 9+ years of experience in data engineering, with a focus on Java/python, Spark and SQL Programming languages. Hands-on experience with Apache Iceberg, Snowflake, or similar technologies. Strong understanding of data lakehouse architectures and data warehousing principles. Proficiency in AWS data services. Experience with version control systems like Git and CI/CD pipelines. Excellent problem-solving and analytical skills. Strong communication and collaboration skills. Nice to Have: Experience with containerization (Docker, Kubernetes) and orchestration tools like Airflow. Certifications in Azure cloud technologies

  • Azure Data Engineer

    2 weeks ago

    Kolkata, India Tata Consultancy Services Full time

    Greetings from TCS! Job Title: Azure Data Engineer Required Skillset: Azure Databricks, Pyspark, SQL, ADF, Data Lake Location: Kolkata Experience Range: 4-6 years Job Description Must-Have** Azure Databricks, Pyspark, SQL, ADF Good To Have: Python, Snowflake Responsibility of / Expectations from the Role: Analyzing and translating business needs into...

  • Azure Data Engineer

    3 weeks ago

    Kolkata, India Tata Consultancy Services Full time

    Greetings from TCS! Job Title: Azure Data Engineer Required Skillset: Azure Databricks, Pyspark, SQL, ADF, Data Lake Location: Kolkata Experience Range: 4-6 years Job Description Must-Have** - Azure Databricks, Pyspark, SQL, ADF Good To Have: - Python, Snowflake Responsibility of / Expectations from the Role: - Analyzing and translating business needs into...

  • Azure Data Engineer

    7 days ago

    kolkata, India Tata Consultancy Services Full time

    Greetings from TCS Job Title: Azure Data Engineer Required Skillset: Azure Databricks, Pyspark, SQL, ADF, Data Lake Location: Kolkata Experience Range: 4-6 years Job Description Must-Have** Azure Databricks, Pyspark, SQL, ADF Good To Have: Python, Snowflake Responsibility of / Expectations from the Role: Analyzing and translating business needs into...

  • Azure Data Engineer

    2 weeks ago

    Kolkata, India Tata Consultancy Services Full time

    Greetings from TCS!Job Title: Azure Data EngineerRequired Skillset: Azure Databricks, Pyspark, SQL, ADF, Data LakeLocation: KolkataExperience Range: 4-6 yearsJob DescriptionMust-Have**- Azure Databricks, Pyspark, SQL, ADFGood To Have:- Python, SnowflakeResponsibility of / Expectations from the Role:- Analyzing and translating business needs into long-term...

  • Azure Architect

    4 weeks ago

    Kolkata, India Polestar Analytics Full time

    Job Title: Azure Architect – Analytics & Data ServicesLocation: Noida/Kolkata/BengaluruJob Type: Full-timeExperience: 8+ yearsIndustry: Analytics ServicesRoles and ResponsibilitiesLead end-to-end architecture design for data and analytics solutions on Azure.Define and implement Azure-based modern data platforms, including data lakes and data...

  • Azure Data Engineer

    2 days ago

    Kolkata, West Bengal, India Capgemini Engineering Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    At Capgemini Invent, we believe difference drives change. As inventive transformation consultants, we blend our strategic, creative and scientific capabilities, collaborating closely with clients to deliver cutting-edge solutions. Join us to drive transformation tailored to our client's challenges of today and tomorrow. Informed and validated by science and...

  • Azure Data Engineer

    1 week ago

    Kolkata, West Bengal, India Sys Edge Micro Informatics Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Role & responsibilitiesAssist in data collection, cleansing, and preparation from various sources.Support basic data modelling and documentation efforts.Develop and maintain simple data pipelines using tools like Azure Data Factory (ADF).Write and optimize queries using SQL.Use Python or PySpark scripts to perform data transformations.Perform data...

  • Data support engineer

    1 week ago

    Kolkata, India Kumaran Systems Full time

    Dear Candidate, Greeting, Experience : 8+ Years Work Model: WFH Work Timings: Starts from 3pm We’re looking for a skilled Data Support Engineer with 8+ years of experience . The ideal candidate will serve as the escalation point for data ingestion, pipeline, and consumption issues while ensuring high system stability through proactive monitoring and...

  • Azure Data Engineer

    3 weeks ago

    Kolkata, India Capgemini Full time

    Job Description At Capgemini Invent, we believe difference drives change. As inventive transformation consultants, we blend our strategic, creative and scientific capabilities,collaborating closely with clients to deliver cutting-edge solutions. Join us to drive transformation tailored to our client's challenges of today and tomorrow.Informed and validated...

  • Azure Data Engineering Specialist

    5 days ago

    kolkata, India beBeeDataEngineering Full time

    Job Title: Azure Data Engineering SpecialistWe are seeking a highly skilled Azure Synapse Analytics engineer with 6+ years of experience to design and implement scalable data pipelines, data warehouses, and ETL/ELT processes. The ideal candidate will have proven experience with Azure Synapse Analytics, Azure Data Factory, and SQL/T-SQL, as well as a strong...