Senior Big Data Engineer

2 months ago


Pune, India AEXONIC TECHNOLOGIES PRIVATE LIMITED Full time

Position - Senior Big Data Engineer (GCP)

Experience - 5 to 8 Yrs

Location - Remote

Shift - EST Shift 5:30 PM to 2:30 AM

We are seeking a highly skilled Senior Big Data Engineer with extensive experience in Big Data technologies, including Hadoop, Hive, PySpark, and expertise in Google Cloud Platform (GCP) and BigQuery. The successful candidate will be responsible for designing, building, and maintaining robust data pipelines that support large-scale data processing and analytics. This role requires excellent data analysis skills and the ability to architect solutions for cloud-based environments, with a focus on extracting, processing, and analyzing data from diverse sources.

Key Responsibilities :

- Design, develop, and optimize Big Data pipelines and workflows using Hadoop, Hive, PySpark, and Google BigQuery.

- Architect data solutions to facilitate smooth data migration and processing between on-premises systems and Google Cloud Platform (GCP).

- Create and manage Unix Shell Scripts for process automation and data handling tasks.

- Perform in-depth data analysis and work comfortably with large datasets to derive actionable insights.

- Collaborate effectively with cross-functional teams to understand data requirements and translate them into scalable data solutions.

- Ensure data quality, integrity, and security across all stages of the data lifecycle.

- Stay updated with the latest industry trends and emerging technologies to continuously improve data solutions.

Mandatory Qualifications :

- Minimum of 8 years of industry experience working with Big Data technologies.

- Proficiency in Hadoop, Hive, and PySpark for data processing.

- Strong experience with Unix Shell Scripting.

- Expertise in SQL and extracting data from multiple sources.

- Demonstrated experience in data analysis, specifically querying and analyzing large datasets on Hadoop HDFS using Hive and Spark.

- Proven experience in architecting Big Data pipelines and data solutions.

- Proficiency in Google BigQuery and migrating data pipelines to GCP.

- Exceptional communication skills for clear requirement gathering and collaboration.

Preferred Qualifications :

- Google Cloud Platform (GCP) certification.

- Professional experience with cloud hosting platforms, with a preference for GCP.

(ref:hirist.tech)

  • Pune, Maharashtra, India NTT DATA Full time

    Company OverviewNTT DATA is a global technology services company that helps clients innovate, optimize and transform for long term success. We have diverse experts in more than 50 countries and a robust partner ecosystem of established and start-up companies.Our Commitment to YouEstimated Salary: ₹2,200,000 - ₹3,500,000 per annum (dependent on...

  • Senior Data Engineer

    6 months ago


    Pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Senior Data Engineer

    4 weeks ago


    Pune, Maharashtra, India TapTalent Full time

    TapTalent is a leading player in the job market, and we're looking for a seasoned Data Engineer to join our team. As a senior member of our engineering team, you will play a crucial role in designing and developing our big data platform.We're looking for someone with at least 4 years of experience in working with Scala Spark. If you have expertise in data...

  • Senior Engineer

    2 weeks ago


    Pune, India Infinity career edge Full time

    Responsibilities :- Works closely with the business owner / Product owner / Cross functional Team and gathering technical requirements.- Experience in building and maintaining reliable and scalable ETL pipeline on big data & / or Cloud platform through the collection, storage, processing, and transformation of large datasets.- Collaborate with other teams to...


  • Pune, Maharashtra, India Tata Consultancy Services Full time

    We are seeking an experienced Senior Data Engineer to work on our Big Data projects. The ideal candidate will have good work experience on platforms like Hadoop, Spark, Scala, Hive, Impala, and SQL.A successful candidate will have hands-on experience working on Data Engineering projects and a good understanding of SQL concepts.The role requires strong...


  • Pune, India AEXONIC TECHNOLOGIES PRIVATE LIMITED Full time

    Are you ready to take your Big Data expertise to the next level? Join our dynamic team at Aexonic Technologies Pvt. Ltd. as a Senior Big Data Engineer (GCP) and work on cutting-edge projects in a collaborative and innovative environment.Role : Senior Big Data Engineer (GCP) Experience : 5+ years in Big Data technologies Shift : EST Shift (5:30 PM - 2:30 AM...


  • Pune, Maharashtra, India PubMatic Full time

    About the RoleWe are seeking an experienced Senior Software Development Engineer to join our team in designing, developing, and testing complex engineering projects. The ideal candidate will have a strong background in software development, testing methodologies, and big data technologies.Key ResponsibilitiesDevelop automation frameworks using Python to...


  • Pune, India RS Consultants Full time

    Job Title : Senior Data Engineer. Location : Pune. Experience Level : 3+ Years. Company Type : AI/ML startup. Please apply only if you are comfortable working with Early stage startup. Company Overview : Join our client a pioneering startup founded by renowned tech experts, marking their second venture in the AI and ML field. The founders previously...

  • Senior Data Engineer

    2 months ago


    Pune, India RS Consultants Full time

    Job Title : Senior Data Engineer. Location : Pune. Experience Level : 3+ Years. Company Type : AI/ML startup. Please apply only if you are comfortable working with Early stage startup. Company Overview : Join our client a pioneering startup founded by renowned tech experts, marking their second venture in the AI and ML field. The founders previously...


  • Pune, Maharashtra, India Tata Consultancy Services Full time

    About the RoleWe are seeking a highly skilled Senior Data Pipeline Engineer to join our team in Pune, Hyderabad, Bangalore, Chennai, Noida or Nagpur. The ideal candidate will have 5-10 years of experience in designing and building production data pipelines using Cloud Native GCP, Python, SQL and other relevant technologies.Key ResponsibilitiesDesign and...


  • Pune, Maharashtra, India Epam Full time

    EPAM is a leading global provider of digital platform engineering and development services.We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture.As a Senior Big Data Solutions Architect, you will play a crucial role in designing and implementing Big Data solutions for our...

  • Big data developer

    3 days ago


    Pune, India Recro Full time

    To be successful in this role, you should possess• Collaborate closely with Product Management and Engineering leadership to devise and build the right solution.• Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data tools and frameworks required to solve Big Data problems at scale.• Design and...


  • Pune, Maharashtra, India Mig Staffing Full time

    We are seeking a highly skilled Senior Big Data Developer to design and implement robust Big Data solutions using Hadoop technologies. As a key member of our team, you will be responsible for developing and managing large-scale data processing systems, collaborating with cross-functional teams, and ensuring data quality and integrity.Key...


  • Pune, Maharashtra, India Recro Full time

    About the RoleWe are seeking a highly skilled Big Data Engineer to join our team at Recro.Job DescriptionIn this role, you will be responsible for designing and implementing scalable data pipelines using distributed processing tools like Akka and Spark.You will work closely with Product Management and Engineering leadership to devise and build the right...

  • Big Data Engineer

    4 weeks ago


    Pune, India Flipped.ai Full time

    Job Description : Big Data Engineer (PySpark)Years of experience : 6 to 13 yearsLocations : Chennai and Pune ( Candidates from chennai and pune only apply)Notice period : Immediate joiners/15 to 30 days onlyAbout the Role : We are seeking a highly skilled and experienced Big Data Engineer with a strong focus on PySpark to join our dynamic team. As a Big Data...

  • Big Data Developer

    5 days ago


    Pune, India Recro Full time

    To be successful in this role, you should possess• Collaborate closely with Product Management and Engineering leadership to devise and build the right solution.• Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data tools and frameworks required to solve Big Data problems at scale.• Design and...

  • Big Data Developer

    4 days ago


    Pune, India Recro Full time

    To be successful in this role, you should possess • Collaborate closely with Product Management and Engineering leadership to devise and build the right solution. • Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data tools and frameworks required to solve Big Data problems at scale. • Design and...

  • Big Data Developer

    3 days ago


    Pune, India Recro Full time

    To be successful in this role, you should possess• Collaborate closely with Product Management and Engineering leadership to devise and build the right solution.• Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data tools and frameworks required to solve Big Data problems at scale.• Design and...

  • Big Data Engineer

    1 month ago


    Chennai/Pune, India Pan Asia Man power Consultants Recruiting agency Full time

    Big Data EngineerWe are seeking a skilled Big Data Engineer to contribute to building backend data applications and services. This role requires expertise in various technologies to drive our data initiatives.Key Responsibilities:Design, develop, and maintain backend data applications and services.Collaborate with product owners and team members to gather...

  • Data Engineer

    4 days ago


    Pune, Maharashtra, India LTIMindtree Full time

    Job DescriptionWe are seeking a highly skilled Data Engineer to join our team at LTIMindtree. As a Data Engineer, you will play a key role in designing and developing big data platforms using Scala and Spark.Responsibilities:Design and develop big data platforms using Scala and Spark.Develop expertise in Data Engineering, including hands-on experience in...