Big Data Engineer
3 weeks ago
:
We are seeking a skilled Big Data Engineer with deep expertise in PySpark, Python, MySQL, Hadoop and Linux. The ideal candidate should have extensive experience deploying and managing these tools within on-premises environments (no public cloud). Proficiency in cluster management, performance optimization, and monitoring is essential for this role.
Key Responsibilities:Develop and implement data ingestion pipelines and ETL processes using
PySpark
and
Python
capable of managing and processing
large volumes
of data, ranging in the
millions
of records.Design and develop
APIs
using Python to facilitate data integration and processing.Deploy, configure, and maintain
Hadoop
ecosystem components including
HDFS , YARN, Hive, and HBase in on-premises environments.Optimize the
performance
of Big Data applications and queries, ensuring efficient resource utilization.Manage and monitor cluster
health , including tuning and
troubleshooting
performance issues.Ensure data
security ,
integrity , and
availability
within the Big Data infrastructure.Collaborate with cross-functional teams to integrate data analytics solutions and support business requirements.
Required Skills and Qualifications:Bachelor’s degree in computer science, Engineering, or a related field;Proven experience 3+ years working with Big Data technologies such as Hadoop, PySpark, and MySQL in on-premises environments and handling large-scale data volumes, including datasets with millions of records, in on-premises environments.Strong programming skills in Python, with experience in developing APIs.Hands-on experience in Linux.Expertise in deploying, managing, and optimizing Big Data clusters in a local data center.Proficiency in monitoring and performance tuning of Hadoop and PySpark clusters.Ability to troubleshoot and resolve complex issues related to data processing and infrastructure.Excellent problem-solving skills and the ability to work independently as well as part of a team.Strong communication skills with the ability to convey technical concepts to non-technical stakeholders.Experience in dockers
Preferred Skills:Experience with Apache Airflow, Kafka, Apache Flink, and Apache NiFi for data orchestration and streaming.Certification in Hadoop or related Big Data technologies.
Application Process:
Interested candidates should submit their resume and a cover letter outlining their qualifications and experience in
Big Data engineering and on-premises cluster management
and include details of at least one relevant project involving the handling of large data volumes
(in millions of records),
you have worked on, including a brief description of the project, your role and the technologies used. This is
mandatory requirement for our screening process
.
-
Big Data Engineer
3 weeks ago
delhi, India Pert Telecom Solutions Full timeJob Description : We are seeking a skilled Big Data Engineer with deep expertise in PySpark, Python, MySQL, Hadoop and Linux. The ideal candidate should have extensive experience deploying and managing these tools within on-premises environments (no public cloud). Proficiency in cluster management, performance optimization, and monitoring is essential for...
-
Big Data Engineer
3 weeks ago
Delhi, India Pert Telecom Solutions Full timeJob Description : We are seeking a skilled Big Data Engineer with deep expertise in PySpark, Python, MySQL, Hadoop and Linux. The ideal candidate should have extensive experience deploying and managing these tools within on-premises environments (no public cloud). Proficiency in cluster management, performance optimization, and monitoring is essential for...
-
Big data engineer
3 weeks ago
Delhi, India Pert Telecom Solutions Full timeJob Description : We are seeking a skilled Big Data Engineer with deep expertise in Py Spark, Python, My SQL, Hadoop and Linux. The ideal candidate should have extensive experience deploying and managing these tools within on-premises environments (no public cloud). Proficiency in cluster management, performance optimization, and monitoring is essential...
-
Big Data Engineer
3 weeks ago
Delhi, India Pert Telecom Solutions Full timeJob Description:We are seeking a skilled Big Data Engineer with deep expertise in PySpark, Python, MySQL, Hadoop and Linux. The ideal candidate should have extensive experience deploying and managing these tools within on-premises environments (no public cloud). Proficiency in cluster management, performance optimization, and monitoring is essential for this...
-
delhi, India Data Warehouse Engineer Full timeExperience : 2- 5 YearsPrimary Skills : Strong SQL knowledge, Data warehouse concepts and ETL hands on experience, Azure services like Data Factory, Azure Data Lake, and Good analysis skills.Good to have skills: Power BI, Databricks, Python.
-
Data Warehouse Engineer
1 week ago
Delhi, India Data Warehouse Engineer Full timeExperience : 2- 5 YearsPrimary Skills : Strong SQL knowledge, Data warehouse concepts and ETL hands on experience, Azure services like Data Factory, Azure Data Lake, and Good analysis skills.Good to have skills: Power BI, Databricks, Python.
-
Big Data engineer
18 hours ago
Delhi, India Tata Consultancy Services Full timeRole ; Big Data engineer Location : Pune, Chennai, MumbaiExp : 3-6 YearRequired Technical Skill Set** Hadoop, Python, PySpark, HIVEMust-Have**· Hands-on experience of Hadoop, Python, PySpark, Hive, Big Data Eco System Tools. · Should be able to develop, tweak queries and work on performance enhancement. · Solid understanding of object-oriented...
-
Big Data Automation Engineer
15 hours ago
Delhi, Delhi, India PubMatic Full timeJob DescriptionWe are seeking a highly skilled Big Data Automation Engineer to join our team at PubMatic. As a key member of our engineering team, you will be responsible for designing, developing, and maintaining automation frameworks for big data workflows.Responsibilities:Designing and implementing automation frameworks using Python to automate big data...
-
Big Data Engineer
2 weeks ago
Delhi, Delhi, India Wipro Full timeWe are seeking a highly skilled Big Data Engineer to join our team at Wipro. As a Big Data Engineer, you will play a key role in designing, developing, and maintaining large-scale data processing systems.The ideal candidate will have a strong background in Hadoop and related technologies such as Scala. You will work closely with cross-functional teams to...
-
Data Engineer for Big Data Solutions
3 days ago
Delhi, Delhi, India LTIMindtree Full timeWe are seeking an experienced Data Engineer to join our team at LTIMindtree. As a key member of our big data solutions group, you will be responsible for designing and developing scalable data platforms using modern technologies like Spark, HBase, Hive, and other Hadoop ecosystem tools.About the RoleIn this role, you will work closely with cross-functional...
-
Big Data Engineer
1 month ago
Delhi, Delhi, India Talent Pro Full timeHadoop Developer Job DescriptionWe are currently looking for a skilled and experienced Hadoop Developer to join our team at Talent Pro.As a Hadoop Developer, you will be responsible for designing, developing, and maintaining large-scale big data processing pipelines using the Hadoop ecosystem (HDFS, MapReduce) and Spark.The ideal candidate will have a strong...
-
Big Data Developer
20 hours ago
Delhi, India Impetus Full timeLead Software EngineerLocations- Indore, Noida, HyderabadJob DescriptionWe are looking for a Lead Big Data Developer to design and oversee the implementation of large-scale data solutions, lead a talented team, and ensure the efficient processing and analysis of massive datasets.Roles & ResponsibilitiesKey Responsibilities- Architect Data Solutions: Design...
-
Engineering Manager
3 weeks ago
Delhi, Delhi, India PROTEGRITY INDIA PRIVATE LIMITED Full timeAbout the RoleProtegrity - Engineering Manager - Big DataProtegrity is a pioneering company in data protection, and we're seeking a seasoned Engineering Manager to lead our Big Data team. This role requires a strategic thinker with expertise in data security and a passion for mentoring and guiding technical teams.Key ResponsibilitiesManage the research and...
-
Cloud Data Engineer
15 hours ago
Delhi, Delhi, India Tata Consultancy Services Full timeJob Role: Cloud Data Engineer - Big Data Solutions SpecialistAbout the Role: Tata Consultancy Services is seeking a skilled Cloud Data Engineer to join our team in Chennai. As a key member of our big data solutions team, you will be responsible for developing and deploying scalable big data pipelines on Google Cloud Platform (GCP).Responsibilities: Develop...
-
Big Data Developer
4 weeks ago
delhi, India Manuh Technologies Full timeBig Data Engineer with 5+ years of experience on technologies Cloudera, Hadoop, sqoop, Spark, PySpark, Impala, SQL Previous experince into big data projects using hadoop and spark. Atleast one project experience into Cloudera hadoop with Impala and Hue Strong Programming experience in Python and PySpark Hands on experience in SQL using Impala Experience in...
-
Senior Data Engineer
2 weeks ago
Delhi, Delhi, India PRONEXUS CONSULTING PRIVATE LIMITED Full time**Job Title:** Senior Data Engineer - Big Data Pipeline SpecialistAt Pronexus Consulting Private Limited, we are seeking an experienced and highly skilled Senior Data Engineer - Big Data Pipeline Specialist to join our team. This role will involve designing, developing, and implementing large-scale data pipelines using various technologies such as Spark,...
-
Big Data Developer
3 weeks ago
delhi, India Manuh Technologies Full timeBig Data Engineer with 5+ years of experience on technologies Cloudera, Hadoop, sqoop, Spark, PySpark, Impala, SQL Previous experince into big data projects using hadoop and spark.Atleast one project experience into Cloudera hadoop with Impala and HueStrong Programming experience in Python and PySparkHands on experience in SQL using ImpalaExperience in...
-
Big Data Developer
4 weeks ago
delhi, India Manuh Technologies Full timeBig Data Engineer with 5+ years of experience on technologies Cloudera, Hadoop, sqoop, Spark, PySpark, Impala, SQLPrevious experince into big data projects using hadoop and spark.Atleast one project experience into Cloudera hadoop with Impala and HueStrong Programming experience in Python and PySparkHands on experience in SQL using ImpalaExperience in...
-
Big Data Developer
4 weeks ago
Delhi, India Manuh Technologies Full timeBig Data Engineer with 5+ years of experience on technologies Cloudera, Hadoop, sqoop, Spark, PySpark, Impala, SQLPrevious experince into big data projects using hadoop and spark.Atleast one project experience into Cloudera hadoop with Impala and HueStrong Programming experience in Python and PySparkHands on experience in SQL using ImpalaExperience in...
-
Data Engineering Expert for Big Data Systems
3 days ago
Delhi, Delhi, India NEXUS CONSULTING SOLUTIONS Full timeWe are seeking a skilled Data Engineer to join our team at NEXUS CONSULTING SOLUTIONS. This role requires experience in working with large data sets and a strong understanding of big data systems.Job DescriptionA successful candidate will have hands-on experience in Python and Scala programming languages. Additionally, they should be familiar with the Spark...