Technical Lead

2 days ago

bangalore district, India Sigmoid Full time

Job Title:Technical Lead- Data Engineer

Location: Bangalore

Years of Experience: 8+ years of experience

Sigmoid works with a variety of clients from start-ups to fortune 500 companies. We are looking for a detailed oriented self-starter to assist our engineering and analytics teams in various roles as a Software Development Engineer.

As an Technical Lead- Data Engineer, you will be responsible for building a highly-scalable and extensible big data platform that provides the foundation for collecting, storing, modeling, and analyzing massive data sets from multiple channels.

We are looking for a skilled Data Engineer with 9+ years of experience in big data technologies, particularly Python, PYSpark, SQL and data lakehouse architectures. The ideal candidate will have a strong background in building scalable data pipelines and experience with modern data storage formats, including Apache Iceberg. You will work closely with cross-functional teams to design and implement efficient data solutions in a cloud-based environment.

Key Responsibilities:

  • Data Pipeline Development:
  • Design, build, and optimize scalable data pipelines using Apache Spark.
  • Implement and manage large-scale data processing solutions across data lakehouses.
  • Data Lakehouse Management:
  • Work with modern data lakehouse platforms (e.g.Apache Iceberg) to handle large datasets.
  • Optimize data storage, partitioning, and versioning to ensure efficient access and querying.
  • SQL & Data Management:
  • Write complex SQL queries to extract, manipulate, and transform data.
  • Develop performance-optimized queries for analytical and reporting purposes.
  • Data Integration:
  • Integrate various structured and unstructured data sources into the lakehouse environment.
  • Work with stakeholders to define data needs and ensure data is available for downstream consumption.
  • Data Governance and Quality:
  • Implement data quality checks and ensure the reliability and accuracy of data.
  • Contribute to metadata management and data cataloging efforts.
  • Performance Tuning:
  • Monitor and optimize the performance of Spark jobs, SQL queries, and overall data infrastructure.
  • Work with cloud infrastructure teams to optimize costs and scale as needed.

Qualifications:

  • Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
  • 8+ years of experience in data engineering, with a focus on Java/python, Spark and SQL Programming languages.
  • Hands-on experience with Apache Iceberg, Snowflake, or similar technologies.
  • Strong understanding of data lakehouse architectures and data warehousing principles.
  • Proficiency in AWS data services.
  • Experience with version control systems like Git and CI/CD pipelines.
  • Excellent problem-solving and analytical skills.
  • Strong communication and collaboration skills.

Nice to Have:

  • Experience with containerization (Docker, Kubernetes) and orchestration tools like Airflow.
  • Certifications in AWS cloud technologies

  • Technical Lead

    1 day ago

    bangalore district, India Fulfillment IQ Full time

    Thank you for your interest in the opportunity at Fulfillment IQ! If this role aligns with your experience and career goals, feel free to share your updated resume at chitrank.rastogi@fulfillmentiq.com. You can also connect with me directly on LinkedIn – Chitrank Rastogi – and I’ll be happy to reach out to discuss the opportunity in more detail....

  • Senior Technical Lead

    1 day ago

    bangalore district, India Perfios Full time

    About the Role: We are seeking a seasoned Senior Technical Lead to drive the efforts of a dynamic, high-performing team of engineers responsible for developing and operating enterprise-grade platforms leveraging Java and Python. The ideal candidate will possess deep architectural expertise, hands-on technical skills, a proven track record in building and...

  • Technical Lead

    2 days ago

    bangalore district, India upGrad Full time

    Job Title: Technical Lead — Backend (Spring Boot / Distributed Systems) Location: Bangalore | Experience: 5+ years | Type: Full-time | Department: Engineering / Technology About the Role: We are seeking a hands-on Technical Lead (Backend) with deep expertise in Java and Spring Boot to lead the design, development, and operation of scalable, secure...

  • Technical Lead

    2 days ago

    bangalore district, India Infogain Full time

    Exp: 8 to 11 years Job Location: Noida / Mumbai / Pune / Bangalore / Gurgaon ( Hybrid work) Job Description: •Implementing complex solutions in Workfront from requirements developed in partnership with peers and delivered in clear layouts for business leaders and non-technical marketers to grasp. •Mapping out and defining complex data solution...

  • Technical Lead-Data Engineer

    1 day ago

    bangalore district, India Best Infosystems Ltd. Full time

    Technical Lead-Data Engineer_Full-Time_Bangalore/Pune/Navi Mumbai/Noida/Hyderabad/Chennai Job Title: Technical Lead-Data Engineer Location: Bangalore/Pune/Navi Mumbai/Noida/Hyderabad/Chennai Job Type: Full-Time Experience: 8+ years Qualifications: Bachelor’s/relevant exp - with Snowpro Certification Job Description: Must Have Skills:...

  • Technical Lead

    1 week ago

    bangalore, India HCLTech Full time

    Technical lead with .net,c#,wpfJob SummaryThe Technical Lead with expertise in .NET, C#, and WPF will be responsible for leading and managing a team of developers in designing, developing, and implementing software solutions. They will be the key point of contact for technical issues, providing guidance and ensuring high-quality code delivery within project...

  • Technical Lead

    1 week ago

    Bangalore, India Scrumconnect Consulting Full time

    About Us At Scrumconnect Consulting, we work with clients to deliver high-quality digital solutions using agile methodologies. We pride ourselves on building strong, collaborative teams and leveraging cutting-edge technology to solve complex business challenges. Position: Technical Lead We are seeking an experienced Technical Lead to join our growing...

  • Technical lead

    5 days ago

    Bangalore, India Scrumconnect Consulting Full time

    About Us At Scrumconnect Consulting, we work with clients to deliver high-quality digital solutions using agile methodologies. We pride ourselves on building strong, collaborative teams and leveraging cutting-edge technology to solve complex business challenges. Position: Technical Lead We are seeking an experienced Technical Lead to join our growing...

  • Technical Lead

    3 days ago

    bangalore, India Green HR Solutions Full time

    Hiring for USA based MNC,We are seeking an experienced and hands-on Technical Lead to guide the design, development, and delivery of software solutions. The role involves overseeing technical teams, ensuring code quality, and aligning technology decisions with business goals. The ideal candidate has deep technical expertise, leadership skills, and the...

  • Technical Lead

    1 week ago

    Bangalore, India Scrumconnect Consulting Full time

    About Us At Scrumconnect Consulting, we work with clients to deliver high-quality digital solutions using agile methodologies. We pride ourselves on building strong, collaborative teams and leveraging cutting-edge technology to solve complex business challenges. Position: Technical Lead We are seeking an experienced Technical Lead to join our...