Senior Big Data Developer

3 weeks ago


Pune, Maharashtra, India DATAECONOMY Full time
Job Overview

We are seeking a highly skilled Senior Python & PySpark Developer to join our team. The ideal candidate will have extensive experience with Python development, PySpark, and working with large datasets in distributed computing environments.

Key Responsibilities:

  • Develop, maintain, and optimize scalable data processing pipelines using Python and PySpark.
  • Collaborate with cross-functional teams to understand business requirements and translate them into technical specifications.
  • Work with large datasets to perform data wrangling, cleansing, and analysis.
  • Implement best practices for efficient distributed computing and data processing.
  • Optimize existing data pipelines for performance and scalability.
  • Conduct code reviews, mentor junior developers, and contribute to team knowledge sharing.
  • Develop and maintain technical documentation.
  • Troubleshoot, debug, and resolve issues related to data processing.
  • Collaborate with data engineers, data scientists, and analysts to deliver high-quality solutions.
Requirements
  • Bachelor's or Master's degree in Computer Science, Data Engineering, or related field.
  • 5 years of experience in Python programming.
  • 3 years of hands-on experience with PySpark and distributed data processing frameworks.
  • Strong understanding of big data ecosystems (Hadoop, Spark, Hive).
  • Experience working with cloud platforms like AWS, GCP, or Azure.
  • Proficient with SQL and relational databases.
  • Familiarity with ETL processes and data pipelines.
  • Strong problem-solving skills with the ability to troubleshoot and optimize code.
  • Excellent communication skills and the ability to work in a team-oriented environment.
Preferred Qualifications
  • Experience with Apache Kafka or other real-time data streaming technologies.
  • Familiarity with machine learning frameworks (TensorFlow, Scikit-learn).
  • Experience with Docker, Kubernetes, or other containerization technologies.
  • Knowledge of DevOps tools (CI/CD pipelines, Jenkins, Git, etc.).
  • Familiarity with data warehousing solutions such as Redshift or Snowflake.
Benefits

$120,000 - $180,000 per year, depending on experience.



  • Pune, Maharashtra, India NTT DATA Full time

    Company OverviewNTT DATA is a global technology services company that helps clients innovate, optimize and transform for long term success. We have diverse experts in more than 50 countries and a robust partner ecosystem of established and start-up companies.Our Commitment to YouEstimated Salary: ₹2,200,000 - ₹3,500,000 per annum (dependent on...


  • Pune, Maharashtra, India Mig Staffing Full time

    We are seeking a highly skilled Senior Big Data Developer to design and implement robust Big Data solutions using Hadoop technologies. As a key member of our team, you will be responsible for developing and managing large-scale data processing systems, collaborating with cross-functional teams, and ensuring data quality and integrity.Key...


  • Pune, Maharashtra, India 12542 Citicorp Services India Private Limited Full time

    Key Responsibilities:As a Senior Big Data Developer at 12542 Citicorp Services India Private Limited, you will be responsible for developing and implementing large-scale data processing systems, ensuring high performance, scalability, and reliability.Requirements:6+ years of experience in programming with SQL, Java, Python, J2EE, Unix/Linux shell, and...


  • Pune, Maharashtra, India 12542 Citicorp Services India Private Limited Full time

    Job Summary We are seeking a highly skilled Big Data Developer to join our team at 12542 Citicorp Services India Private Limited. As a Big Data Developer, you will be responsible for designing, developing, and implementing big data solutions using various technologies such as Apache Hadoop, Spark, Hive, Impala, Yarn, and Talend.Key Responsibilities •...


  • Pune, Maharashtra, India Synechron Full time

    Job Overview Synechron is a global consulting firm that combines creativity and innovative technology to deliver industry-leading digital solutions. We are seeking an experienced Senior Big Data Developer to join our team in Pune, India.About the Role As a Senior Big Data Developer, you will be responsible for designing and developing data-driven solutions...


  • Pune, Maharashtra, India Synechron Full time

    Job Title:Sr. Big Data DeveloperAbout the Job:We are seeking a highly skilled Sr. Big Data Developer to join our team at Synechron. The successful candidate will have a strong background in Python and experience in designing and developing enterprise-level big data solutions.Key Responsibilities:Design and develop complex big data solutions using PythonWork...


  • Pune, Maharashtra, India Capgemini Full time

    Job SummaryWe are seeking a Senior Big Data Developer Lead to join our team at Capgemini.About the RoleThe ideal candidate will have hands-on experience with Scala and object-oriented programming concepts, as well as distributed frameworks like Spark. They should be proficient in Spark RDD, Spark Dataframe, Spark Core, and Spark Streaming.A strong...


  • Pune, Maharashtra, India PubMatic Full time

    About the RoleWe are seeking an experienced Senior Software Development Engineer to join our team in designing, developing, and testing complex engineering projects. The ideal candidate will have a strong background in software development, testing methodologies, and big data technologies.Key ResponsibilitiesDevelop automation frameworks using Python to...


  • Pune, Maharashtra, India Absolutdata Full time

    About the RoleWe are looking for a talented Senior Big Data Architect to join our team at Absolutdata. In this role, you will be responsible for designing and implementing big data infrastructure, working closely with the operations and analytics teams to understand data landscaping requirements and build the systems, processes, and teams required.Key...


  • Pune, Maharashtra, India Epam Full time

    EPAM is a leading global provider of digital platform engineering and development services.We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture.As a Senior Big Data Solutions Architect, you will play a crucial role in designing and implementing Big Data solutions for our...

  • Senior Data Engineer

    4 weeks ago


    Pune, Maharashtra, India PubMatic Full time

    PubMatic is an innovative technology company that empowers digital content creators to control their inventory and increase monetization.Our sell-side platform enables marketers to drive return on investment and reach addressable audiences across ad formats and devices.We champion a vibrant and transparent digital advertising supply chain.As a Senior Data...


  • Pune, Maharashtra, India Talentmatics Full time

    Company Overview:Talentmatics is a leading provider of innovative solutions for the digital age. We are committed to delivering exceptional services and fostering a culture of excellence.Job Summary:We are seeking an experienced Senior Big Data Administrator to join our team in Pune or Hyderabad. This role will be responsible for administering our Big Data...

  • Senior Data Engineer

    3 weeks ago


    Pune, Maharashtra, India TapTalent Full time

    TapTalent is a leading player in the job market, and we're looking for a seasoned Data Engineer to join our team. As a senior member of our engineering team, you will play a crucial role in designing and developing our big data platform.We're looking for someone with at least 4 years of experience in working with Scala Spark. If you have expertise in data...


  • Pune, Maharashtra, India Atyeti Inc Full time

    About Atyeti Inc">Ayetti Inc is a leading company in the field of big data and cloud migration, with a strong focus on innovation and excellence.">Job Overview">We are currently seeking a highly skilled Senior Java Developer to join our team in Pune/Hyderabad/Chennai/Bangalore/ Trivandrum. The successful candidate will be responsible for leading the...

  • Data Engineer Leader

    1 month ago


    Pune, Maharashtra, India DATAECONOMY Full time

    About the RoleDataEconomy is seeking a Senior Data Engineer to lead our team in designing and implementing big data solutions using PySpark.Key ResponsibilitiesLead the design and implementation of scalable and robust data pipelines using PySpark.Collaborate with cross-functional teams to understand business needs and develop data-driven solutions.Mentor and...


  • Pune, Maharashtra, India Mig Staffing Full time

    Job Title: Big Data Solutions DeveloperJob Description:Mig Staffing is seeking a Big Data Solutions Developer to design, implement, and manage our company's big data solutions. Key responsibilities include developing Hadoop systems, loading disparate data sets, conducting pre-processing services, and collaborating with software research and development...


  • Pune, Maharashtra, India Tata Consultancy Services Full time

    We are seeking an experienced Senior Data Engineer to work on our Big Data projects. The ideal candidate will have good work experience on platforms like Hadoop, Spark, Scala, Hive, Impala, and SQL.A successful candidate will have hands-on experience working on Data Engineering projects and a good understanding of SQL concepts.The role requires strong...


  • Pune, Maharashtra, India Tata Consultancy Services Full time

    About the RoleWe are seeking a highly skilled Senior Data Pipeline Engineer to join our team in Pune, Hyderabad, Bangalore, Chennai, Noida or Nagpur. The ideal candidate will have 5-10 years of experience in designing and building production data pipelines using Cloud Native GCP, Python, SQL and other relevant technologies.Key ResponsibilitiesDesign and...


  • Pune, Maharashtra, India NTT DATA Full time

    Job SummaryWe are seeking a skilled Systems Integration Senior Specialist to join our team in India. The ideal candidate will have a Master's or Bachelor's degree in computer science or a related field and at least 8 years of experience in implementing data applications and big data platforms.Expertise in designing, engineering, and deploying data ingestion,...


  • Pune, Maharashtra, India Data Axle Full time

    About Data Axle:Data Axle Inc. is a 50-year-old industry leader in data, marketing solutions, sales, and research in the US.We have set up a strategic global centre of excellence in Pune, delivering mission-critical data services to our global customers powered by our proprietary cloud-based technology platform and leveraging our proprietary business &...