Azure Data Engineer-Technical Lead

3 days ago

Bengaluru, India Sigmoid Full time

Kindly find the Job Description Below. Job Title:Technical Lead-Azure Data Engineer Location: Bangalore Years of Experience: 10+ years of experience Sigmoid works with a variety of clients from start-ups to fortune 500 companies. We are looking for a detailed oriented self-starter to assist our engineering and analytics teams in various roles as a Software Development Engineer. As an Technical Lead- Data Engineer, you will be responsible for building a highly-scalable and extensible big data platform that provides the foundation for collecting, storing, modeling, and analyzing massive data sets from multiple channels. We are looking for a skilled Data Engineer with 10+ years of experience in big data technologies, particularly Azure, Python, PYSpark, SQL and data lakehouse architectures. The ideal candidate will have a strong background in building scalable data pipelines and experience with modern data storage formats, including Apache Iceberg. You will work closely with cross-functional teams to design and implement efficient data solutions in a cloud-based environment. Key Responsibilities: - Data Pipeline Development: - Design, build, and optimize scalable data pipelines using Apache Spark. - Implement and manage large-scale data processing solutions across data lakehouses. - Data Lakehouse Management: - Work with modern data lakehouse platforms (e.g.Apache Iceberg) to handle large datasets. - Optimize data storage, partitioning, and versioning to ensure efficient access and querying. - SQL & Data Management: - Write complex SQL queries to extract, manipulate, and transform data. - Develop performance-optimized queries for analytical and reporting purposes. - Data Integration: - Integrate various structured and unstructured data sources into the lakehouse environment. - Work with stakeholders to define data needs and ensure data is available for downstream consumption. - Data Governance and Quality: - Implement data quality checks and ensure the reliability and accuracy of data. - Contribute to metadata management and data cataloging efforts. - Performance Tuning: - Monitor and optimize the performance of Spark jobs, SQL queries, and overall data infrastructure. - Work with cloud infrastructure teams to optimize costs and scale as needed. Qualifications: - Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field. - 9+ years of experience in data engineering, with a focus on Java/python, Spark and SQL Programming languages. - Hands-on experience with Apache Iceberg, Snowflake, or similar technologies. - Strong understanding of data lakehouse architectures and data warehousing principles. - Proficiency in AWS data services. - Experience with version control systems like Git and CI/CD pipelines. - Excellent problem-solving and analytical skills. - Strong communication and collaboration skills. Nice to Have: - Experience with containerization (Docker, Kubernetes) and orchestration tools like Airflow. - Certifications in Azure cloud technologies

  • Azure Data Engineer-Technical Lead

    3 days ago

    Bengaluru, India Sigmoid Full time

    Kindly find the Job Description Below.Job Title:Technical Lead-Azure Data EngineerLocation: BangaloreYears of Experience: 10+ years of experienceSigmoid works with a variety of clients from start-ups to fortune 500 companies. We are looking for a detailed oriented self-starter to assist our engineering and analytics teams in various roles as a Software...

  • Azure Data Engineer-Technical Lead

    2 days ago

    Bengaluru, India Sigmoid Full time

    Kindly find the Job Description Below.Job Title:Technical Lead-Azure Data EngineerLocation: BangaloreYears of Experience: 10+ years of experienceSigmoid works with a variety of clients from start-ups to fortune 500 companies. We are looking for a detailed oriented self-starter to assist our engineering and analytics teams in various roles as a Software...

  • Azure Data Engineer-Technical Lead

    23 hours ago

    Bengaluru, India Sigmoid Full time

    Kindly find the Job Description Below.Job Title:Technical Lead-Azure Data EngineerLocation: BangaloreYears of Experience: 10+ years of experienceSigmoid works with a variety of clients from start-ups to fortune 500 companies. We are looking for a detailed oriented self-starter to assist our engineering and analytics teams in various roles as a Software...

  • Azure Data Engineer-Technical Lead

    2 days ago

    Bengaluru, India Sigmoid Full time

    Kindly find the Job Description Below.Job Title:Technical Lead-Azure Data EngineerLocation: BangaloreYears of Experience: 10+ years of experienceSigmoid works with a variety of clients from start-ups to fortune 500 companies. We are looking for a detailed oriented self-starter to assist our engineering and analytics teams in various roles as a Software...

  • Technical Lead-Azure Data Engineer

    6 days ago

    Bengaluru, Mumbai, Pune, India Biz Hr It Solutions Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Job Title: Technical Lead-Azure Data EngineerExperience: 6 to 8Years Shift Timings: 12 PM 9 PM IST / 2 PM – 11 PM ISTAbout the RoleWe are looking for a Data Engineer who is passionate about building and maintaining large-scale, high-performance data systems in the cloud. The ideal candidate will have hands-on experience working with modern data platforms,...

  • Data Engineer Technical Lead(Azure)

    13 hours ago

    Bengaluru, India Sigmoid Full time

    Sigmoid empowers enterprises to make smarter, data-driven decisions by blending advanced data engineering with AI consulting. We collaborate with some of the world’s leading data-rich organizations across sectors such as CPG-retail, BFSI, life sciences, manufacturing, and more to solve complex business challenges. Our global team specializes in cloud data...

  • Data Engineer Technical Lead(Azure)

    1 day ago

    Greater Bengaluru Area, India Sigmoid Full time

    Sigmoid empowers enterprises to make smarter, data-driven decisions by blending advanced data engineering with AI consulting. We collaborate with some of the world’s leading data-rich organizations across sectors such as CPG-retail, BFSI, life sciences, manufacturing, and more to solve complex business challenges. Our global team specializes in cloud data...

  • Data Engineer Technical Lead(Azure)

    1 day ago

    Greater Bengaluru Area, India Sigmoid Full time

    Sigmoid empowers enterprises to make smarter, data-driven decisions by blending advanced data engineering with AI consulting. We collaborate with some of the world’s leading data-rich organizations across sectors such as CPG-retail, BFSI, life sciences, manufacturing, and more to solve complex business challenges. Our global team specializes in cloud data...

  • Azure Data Migration

    1 week ago

    Bengaluru, India NTT DATA Full time

    Job Description Req ID:     NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Azure Data Migration to join our team in Bangalore, Karnātaka (IN-KA), India (IN). Job Duties: Primary...

  • Azure Data Engineer

    1 day ago

    Bengaluru, India Tata Consultancy Services Full time

    Role - Azure Data Engineer Experience - 6 to 8 Location - Pan India Required Technical Skill Set** - ADLS/ADF/ADB Desired Competencies : 1.Lead Back-end development & maintenance of Data Quality Product 2. Design and develop Data Pipeline using ADF, Databricks and integrate with other Azure Services 3. Experience in setting up DevOps pipeline 4. Databricks...