Senior Big Data Engineer Specialist

2 weeks ago


India IGT Full time
Job Description

Exciting opportunity for a Senior Big Data Engineer to design, develop, and maintain scalable data pipelines using Python and PySpark.

Key Responsibilities:

  • Design and implement efficient data architectures using AWS services such as S3, Glue, Lambda, and Athena.
  • Work with SQL and MongoDB to manage and query large datasets, ensuring data quality and integrity.
  • Develop and implement solutions using Snowflake for data warehousing and business intelligence.
  • Collaborate with cross-functional teams to gather and understand requirements, leveraging expertise in big data technologies and frameworks.
  • Implement CI/CD pipelines to streamline the deployment of data solutions, utilizing tools such as Jenkins or GitLab.
Requirements

The ideal candidate will possess:

  • Bachelor's degree in Computer Science, Information Technology, or a related field.
  • At least 3 years of experience in data engineering or a similar role, with a strong background in Python and PySpark.
  • Experience with SQL and NoSQL databases, particularly MongoDB, and familiarity with Snowflake and its ecosystem.
  • Hands-on experience with AWS services, including S3, Glue, Lambda, and Athena.
  • Understanding of CI/CD processes and tools, with excellent problem-solving skills and attention to detail.
  • Strong communication and collaboration skills, with the ability to work effectively in a team environment.
Preferred Qualifications

Candidates with additional qualifications may be considered, including:

  • Experience with other cloud platforms (e.g., Azure, Google Cloud Platform).
  • Familiarity with data visualization tools and techniques.
Estimated Salary

$120,000 - $180,000 per year, depending on location and experience.

IGT Solutions provides equal employment opportunities to all individuals based on job-related qualifications and ability to perform a job, without regard to age, gender, gender identity, sexual orientation, race, color, religion, creed, national origin, disability, genetic information, veteran status, citizenship or marital status.



  • india Senior Data Integration Engineer Full time

    Must Have Skills/Skill Requirement:Design and architect integration solutions to connect various enterprise applications, systems, and databases.Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications.Utilize Azure Integration Services such as Azure Logic Apps,...


  • india Senior Data Integration Engineer Full time

    Must Have Skills/Skill Requirement: Design and architect integration solutions to connect various enterprise applications, systems, and databases. Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications. Utilize Azure Integration Services such as Azure Logic Apps,...


  • India Senior Data Integration Engineer Full time

    Must Have Skills/Skill Requirement: Design and architect integration solutions to connect various enterprise applications, systems, and databases. Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications. Utilize Azure Integration Services such as Azure Logic...


  • Anywhere in India/Multiple Locations US Technology International Pvt Ltd Full time

    Job Title: Senior Big Data EngineerJob Summary:US Technology International Pvt Ltd is seeking a Senior Big Data Engineer to lead the development and implementation of big data solutions. The ideal candidate will have expertise in Hadoop, Spark, and other big data technologies, as well as strong leadership and communication skills.Key Responsibilities:Design,...


  • India Atyeti Inc Full time

    At Atyeti Inc, we are seeking a highly skilled Senior Big Data Architect to join our team. In this role, you will be responsible for analyzing and processing large datasets using big data technologies and programming languages such as Scala and Java.About the JobThe ideal candidate should have at least 6 years of experience in high-tech industry and/or IT...


  • india Big Infrastructures and Projects Full time

    Company DescriptionBIG INTERIORS is an Architectural and Interior Design Firm based at Coimbatore and Chennai. We have delivered a number of Creative Interior Solution, both residential and Commercial starting from concept to handover.We are a multicultural team of Passionate individuals dedicated to delivering one of a kind projects. This is always...


  • india Big Infrastructures and Projects Full time

    Company Description BIG INTERIORS is an Architectural and Interior Design Firm based at Coimbatore and Chennai. We have delivered a number of Creative Interior Solution, both residential and Commercial starting from concept to handover. We are a multicultural team of Passionate individuals dedicated to delivering one of a kind projects. This is always...


  • India QntmLogic, LLC. Full time

    Job DescriptionWe are seeking a highly skilled Senior Python Developer for our Big Data development team at QntmLogic, LLC. in Bangalore/Pune (Hybrid). The ideal candidate will have hands-on experience with PySpark, Hadoop, Tableau, and SQL Server development.The successful candidate will be responsible for designing and building big data solutions using...


  • India Syntasa Full time

    About SyntasaSyntasa is a leading provider of software development and support services with operations in the US, Europe, and the APAC region. Our India subsidiary, Syntasa Technologies India Private Limited, offers an exciting opportunity to be part of our world-class development and support center.Job OverviewWe are seeking a highly skilled Big Data...

  • Big Data Engineer

    1 month ago


    India Pert Telecom Solutions Full time

    Job Description : We are seeking a skilled Big Data Engineer with deep expertise in PySpark, Python, MySQL, Hadoop and Linux. The ideal candidate should have extensive experience deploying and managing these tools within on-premises environments (no public cloud). Proficiency in cluster management, performance optimization, and monitoring is essential...


  • india Data Warehouse Engineer Full time

    Experience : 2- 5 YearsPrimary Skills : Strong SQL knowledge, Data warehouse concepts and ETL hands on experience, Azure services like Data Factory, Azure Data Lake, and Good analysis skills.Good to have skills: Power BI, Databricks, Python.


  • India Data Warehouse Engineer Full time

    Experience : 2- 5 Years Primary Skills : Strong SQL knowledge, Data warehouse concepts and ETL hands on experience, Azure services like Data Factory, Azure Data Lake, and Good analysis skills. Good to have skills: Power BI, Databricks, Python.


  • Anywhere in India/Multiple Locations D-TechWorks Pvt Ltd Full time

    About the RoleWe are seeking an experienced Senior Big Data Engineer for Cloud Migration to join our team at D-TechWorks Pvt Ltd.Job SummaryThe ideal candidate will have 7-9 years of experience in leading big data engineering projects using AWS, Azure, or GCP. They should be proficient in BigData Python, PySpark, AWS Glue, Airflow, and Redshift.Key...


  • India NEXUS CONSULTING SOLUTIONS Full time

    Nexus Consulting Solutions seeks an experienced Big Data Systems Engineer to join our team. This is a challenging and rewarding role that requires expertise in designing, developing, and maintaining large-scale big data systems.We are looking for someone with hands-on experience in Python and Scala programming languages. Familiarity with the Spark framework,...


  • India ITI Data Full time

    We are seeking an experienced PySpark Data Engineer to join our team at ITI Data. As a key member of our team, you will design and build scalable data solutions for one of our Fortune 500 Client programs.About the Role:The ideal candidate will have extensive experience in working with Hadoop, NoSQL databases, and Spark framework on Python. You will be...

  • Big Data Developer

    2 weeks ago


    India Luxoft Full time

    Senior Data Engineer Job DescriptionWe are seeking a highly skilled Senior Data Engineer to join our team at Luxoft. As a key member of our Data Engineering & Analytics team, you will play a crucial role in developing data-driven solutions that drive business growth.About the RoleThis is an exciting opportunity for a seasoned data engineer to leverage their...


  • India Data Warehouse Engineer Full time

    About the RoleWe are seeking an experienced Data Warehouse Engineer to join our team. In this role, you will design, develop, and maintain data warehouses that support business intelligence and analytics initiatives.


  • Anywhere in India/Multiple Locations Paralleldots Full time

    Job Description : As a Senior Data Engineer at ParallelDots, you will be responsible for overseeing the design and development of advanced data solutions. You will work closely with data scientists, analysts, and other stakeholders to ensure that our data infrastructure meets the highest standards of scalability, reliability, and : - Lead the design,...

  • Big Data Developer

    2 weeks ago


    India Luxoft Full time

    Project Description: As a Senior Engineer in the Data Engineering & Analytics team, you will develop data & analytics solutions that sit atop vast datasets gathered by retail stores, restaurants, banks, and other consumer-focused companies. The challenge will be to create high-performance algorithms, cutting-edge analytical techniques including machine...

  • Big Data Developer

    2 weeks ago


    India Luxoft Full time

    Project Description:As a Senior Engineer in the Data Engineering & Analytics team, you will develop data & analytics solutions that sit atop vast datasets gathered by retail stores, restaurants, banks, and other consumer-focused companies. The challenge will be to create high-performance algorithms, cutting-edge analytical techniques including machine...