Data Engineer-Technical Lead

4 days ago

bangalore district, India Sigmoid Full time

Kindly find the Job Description Below. Job Title:Technical Lead-Azure Data Engineer Location: Bangalore Years of Experience: 10+ years of experience Sigmoid works with a variety of clients from start-ups to fortune 500 companies. We are looking for a detailed oriented self-starter to assist our engineering and analytics teams in various roles as a Software Development Engineer. As an Technical Lead- Data Engineer, you will be responsible for building a highly-scalable and extensible big data platform that provides the foundation for collecting, storing, modeling, and analyzing massive data sets from multiple channels. We are looking for a skilled Data Engineer with 10+ years of experience in big data technologies, particularly Azure, Python, PYSpark, SQL and data lakehouse architectures. The ideal candidate will have a strong background in building scalable data pipelines and experience with modern data storage formats, including Apache Iceberg. You will work closely with cross-functional teams to design and implement efficient data solutions in a cloud-based environment. Key Responsibilities: Data Pipeline Development: Design, build, and optimize scalable data pipelines using Apache Spark. Implement and manage large-scale data processing solutions across data lakehouses. Data Lakehouse Management: Work with modern data lakehouse platforms (e.g.Apache Iceberg) to handle large datasets. Optimize data storage, partitioning, and versioning to ensure efficient access and querying. SQL & Data Management: Write complex SQL queries to extract, manipulate, and transform data. Develop performance-optimized queries for analytical and reporting purposes. Data Integration: Integrate various structured and unstructured data sources into the lakehouse environment. Work with stakeholders to define data needs and ensure data is available for downstream consumption. Data Governance and Quality: Implement data quality checks and ensure the reliability and accuracy of data. Contribute to metadata management and data cataloging efforts. Performance Tuning: Monitor and optimize the performance of Spark jobs, SQL queries, and overall data infrastructure. Work with cloud infrastructure teams to optimize costs and scale as needed. Qualifications: Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field. 9+ years of experience in data engineering, with a focus on Java/python, Spark and SQL Programming languages. Hands-on experience with Apache Iceberg, Snowflake, or similar technologies. Strong understanding of data lakehouse architectures and data warehousing principles. Proficiency in AWS data services. Experience with version control systems like Git and CI/CD pipelines. Excellent problem-solving and analytical skills. Strong communication and collaboration skills. Nice to Have: Experience with containerization (Docker, Kubernetes) and orchestration tools like Airflow. Certifications in Azure cloud technologies

  • Lead Data Engineer

    7 days ago

    bangalore, India Eucloid Data Solutions Full time

    Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure on Databricks for our clients by participating in activities which may include starting from up- stream and down-stream technology selection to designing and building of...

  • Lead Data Engineer

    6 days ago

    bangalore, India Eucloid Data Solutions Full time

    Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure on Databricks for our clients by participating in activities which may include starting from up- stream and down-stream technology selection to designing and building of...

  • Technical Lead-Data Engineer

    2 weeks ago

    Bangalore, India Best Infosystems Ltd. Full time

    Technical Lead-Data Engineer_Full-Time_Bangalore/Pune/Navi Mumbai/Noida/Hyderabad/Chennai Job Title: Technical Lead-Data Engineer Location: Bangalore/Pune/Navi Mumbai/Noida/Hyderabad/Chennai Job Type: Full-Time Experience: 8+ years Qualifications: Bachelor’s/relevant exp - with Snowpro Certification Job Description: Must Have Skills: •Overall – 8+...

  • Technical Lead-Data Engineer

    2 weeks ago

    bangalore, India Best Infosystems Ltd. Full time

    Technical Lead-Data Engineer_Full-Time_Bangalore/Pune/Navi Mumbai/Noida/Hyderabad/ChennaiJob Title: Technical Lead-Data EngineerLocation: Bangalore/Pune/Navi Mumbai/Noida/Hyderabad/ChennaiJob Type: Full-TimeExperience: 8+ yearsQualifications: Bachelor’s/relevant exp - with Snowpro CertificationJob Description:Must Have Skills:•Overall – 8+...

  • Lead Data Architect

    2 weeks ago

    Bangalore, Karnataka, India NTT DATA Full time

    NTT DATA strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a Lead Data Architect to join our team in Bangalore Karnxc4x81taka IN-KA India IN Position Overview We are seeking a highly...

  • Azure Data Engineer Lead

    2 weeks ago

    bangalore district, India Creative Synergies Group Full time

    Creative Synergies Group Overview Creative is a leading Global Technology Services firm delivering innovative solutions that build upon synergies between our Deep Engineering Domain Expertise and Digital Engineering Services We work with clients across the Global Transportation, Energy/Process, Industrial Products, Hi-Tech, and EdTech verticals Creative’s...

  • Data Engineer-Technical Lead

    6 days ago

    bangalore, India Sigmoid Full time

    Kindly find the Job Description Below.Job Title:Technical Lead-Azure Data EngineerLocation: BangaloreYears of Experience: 10+ years of experienceSigmoid works with a variety of clients from start-ups to fortune 500 companies. We are looking for a detailed oriented self-starter to assist our engineering and analytics teams in various roles as a Software...

  • Data Engineer-Technical Lead

    5 days ago

    bangalore, India Sigmoid Full time

    Kindly find the Job Description Below. Job Title:Technical Lead-Azure Data Engineer Location: Bangalore Years of Experience: 10+ years of experience Sigmoid works with a variety of clients from start-ups to fortune 500 companies. We are looking for a detailed oriented self-starter to assist our engineering and analytics teams in various roles as a Software...

  • Lead data engineer

    2 weeks ago

    Bangalore, India ThinkWise Consulting LLP Full time

    Lead Data Engineer Experience - 10+ Years Location - Bengaluru Hybrid Model We are seeking a Lead Data Engineer to serve as the technical lead for the team responsible for managing and engineering customer data. The ideal candidate will have a strong background in data engineering and software development, with hands-on experience building scalable...

  • Lead Technical Recruiter

    4 days ago

    bangalore district, India Hellowork Consultants Full time

    Job Title: Lead Technical Recruiter Company: Hellowork Consultants Location: Bangalore Job Type: Full-Time | Onsite Experience Level: 5+ Years About Hellowork Consultants Hellowork Consultants is a leading talent acquisition firm specializing in connecting top-tier technical talent with innovative organizations. We work with startups, scale-ups, and...