Big Data Developer

4 weeks ago

Chennai, India IntraEdge Full time

Job Title : Senior Big Data Engineer (Python)

Experience : 5+ Years

Locations : Chennai, Bangalore, Gurugram

Employment Type : Full-Time

Notice Period : Immediate to 30 Days Preferred

Job Summary

We are seeking an experienced Big Data Engineer with strong expertise in Python to design, build, and manage large-scale data pipelines and analytics solutions. The ideal candidate will have hands-on experience working with Big Data technologies and cloud platforms, and a passion for writing efficient, scalable, and maintainable code.

Key Responsibilities

  • Design, develop, and maintain scalable big data pipelines using Python and other data processing tools.
  • Work with distributed data processing frameworks like Spark, Hadoop, Hive, or similar.
  • Implement ETL processes for structured and unstructured data from various sources.
  • Collaborate with data scientists, analysts, and other engineering teams to understand data needs.
  • Optimize data workflows and ensure data quality, integrity, and security.
  • Deploy solutions on cloud platforms (AWS/GCP/Azure) and automate data workflows.
  • Monitor data pipelines, troubleshoot issues, and ensure high availability and performance.

Required Skills & Qualifications

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
  • Minimum 5 years of hands-on experience in Big Data engineering roles.
  • Strong proficiency in Python for data processing and scripting.
  • Experience with Apache Spark , Hadoop ecosystem (Hive, HDFS, HBase), Kafka.
  • Solid understanding of data warehousing concepts and data modeling.
  • Hands-on experience with SQL and NoSQL databases.
  • Familiarity with cloud data platforms like AWS (EMR, S3, Glue) , Azure , or Google Cloud .
  • Good understanding of CI/CD, containerization (Docker, Kubernetes), and version control (Git).

Preferred Skills (Nice to Have)

  • Experience with Airflow or other workflow orchestration tools.
  • Knowledge of real-time data processing (e.g., Spark Streaming, Flink).
  • Exposure to data governance, data lineage, and catalog tools.

Why Join Us?

  • Opportunity to work on cutting-edge Big Data solutions with global clients.
  • Collaborative and innovation-driven work culture.
  • Competitive compensation and career growth opportunities.
  • Flexible work environment with hybrid (based on role).

  • Big data developer

    3 weeks ago

    Chennai, India IntraEdge Full time

    Job Title : Senior Big Data Engineer (Python)Experience : 5+ YearsLocations : Chennai, Bangalore, GurugramEmployment Type : Full-TimeNotice Period : Immediate to 30 Days PreferredJob SummaryWe are seeking an experienced Big Data Engineer with strong expertise in Python to design, build, and manage large-scale data pipelines and analytics solutions. The ideal...

  • Big Data Developer

    1 day ago

    Chennai, Tamil Nadu, India Net2Source Full time ₹ 2,50,000 - ₹ 7,50,000 per year

    Big Data Developer Skills Required: Big Data & Hadoop Ecosystem, Python, PySpark Experience: 5-10 Years Share your updated resume now.

  • Big Data Developer

    4 weeks ago

    Chennai, India IntraEdge Full time

    Job Title: Senior Big Data Engineer (Python)Experience: 5+ YearsLocations: Chennai, Bangalore, GurugramEmployment Type: Full-TimeNotice Period: Immediate to 30 Days PreferredJob SummaryWe are seeking an experienced Big Data Engineer with strong expertise in Python to design, build, and manage large-scale data pipelines and analytics solutions. The ideal...

  • Big Data Developer

    4 weeks ago

    Chennai, India Chiselon Technologies Pvt Ltd Full time

    Hiring!!! We are hiring for one of our client!!! Position - Big Data Engineer Experience - 5+years Location - Chennai, Bangalore Work Mode - Hybrid Notice Period - Immediate Joiners(Seving Notice Period within 15days only) Job Overview We are seeking a skilled Big Data Developer to join our team in a mid-level position. This full-time , hybrid ...

  • Big Data Developer

    4 weeks ago

    Chennai, India Virtusa Full time

    Big Data Developer - CREQ Description Job Description: 5+ years of hands-on experience in designing, building and supporting Data Applications using Spark, Sqoop and Hive Bachelors or masters degree in Computer Science or related field Strong knowledge of working with large data sets and high-capacity big data processing platform Strong experience in...

  • Big Data Developer

    4 weeks ago

    Chennai, India Virtusa Full time

    Big Data Developer - CREQ Description Job Description: 5+ years of hands-on experience in designing, building and supporting Data Applications using Spark, Sqoop and Hive Bachelors or masters degree in Computer Science or related field Strong knowledge of working with large data sets and high-capacity big data processing platform Strong experience in...

  • Senior Big Data Developer

    1 week ago

    Chennai, Tamil Nadu, India MOURI Tech Foundation Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Looking for a Senior Big Data Developer to design, build, and optimize data pipelines and analytics solutions. Candidates with prior exposure to the Ericsson Wallet Platform will be given preference.Key ResponsibilitiesDevelop, test, and deploy high-performance data pipelines using Hadoop, Spark, Kudu, and HBase.Implement ETL/ELT workflows and ensure data...

  • Big Data Developer

    1 week ago

    Bengaluru, Chennai, India Terrabit Consulting Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Skills required: Pyspark, Python, SQL/ Scala and Unix scriptingExperience in Big Data technologiesExperience in writing SQLsHaving work experience in Control-M/Airflow and DevOps will be advantageGood to have knowledge on Banking & Money Laundering .Should have excellent programming scripts, Debugging and analytical thinking.Experience across all stages of...

  • Big Data Developer

    3 weeks ago

    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    4+ years of hands on development experience in programming languages such as JAVA,SCALA using Maven, Apache Spark Frameworks and Unix Shell scriptingShould be comfortable with Unix File system as well as HDFS commandsShould have worked on query languages such as Oracle SQL, Hive SQL, Spark SQL, Impala, HBase DB Should be flexibleShould have good...

  • Big Data Developer

    3 weeks ago

    Chennai, Tamil Nadu, India, Tamil Nadu IntraEdge Full time

    Job Title: Senior Big Data Engineer (Python)Experience: 5+ YearsLocations: Chennai, Bangalore, GurugramEmployment Type: Full-TimeNotice Period: Immediate to 30 Days PreferredJob SummaryWe are seeking an experienced Big Data Engineer with strong expertise in Python to design, build, and manage large-scale data pipelines and analytics solutions. The ideal...