
Big Data Engineer
2 weeks ago
Big Data Engineer (PySpark)
Location: Pune/Nagpur (WFO)
Experience: 8 - 12 Years
Employment Type: Full-time
Job Overview
We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance and reliability.
Key Responsibilities
- Design, develop, and maintain data pipelines using PySpark and related Big Data technologies.
- Work with HDFS, Hive, Sqoop, and other tools in the Hadoop ecosystem.
- Write efficient HiveQL and SQL queries to handle large-scale datasets.
- Perform performance tuning and optimization of distributed data systems.
- Collaborate with cross-functional teams in an Agile environment to deliver high-quality solutions.
- Manage and schedule workflows using Apache Airflow or Oozie.
- Troubleshoot and resolve issues in data pipelines to ensure reliability and accuracy.
Required Skills
- Proven experience in Big Data Engineering with a focus on PySpark.
- Strong knowledge of HDFS, Hive, Sqoop, and related tools.
- Proficiency in SQL/HiveQL for large datasets.
- Expertise in performance tuning and optimization of distributed systems.
- Familiarity with Agile methodology and collaborative team practices.
- Experience with workflow orchestration tools (Airflow/Oozie).
- Strong problem-solving, analytical, and communication skills.
Good to Have
- Knowledge of data modeling and data warehousing concepts.
- Exposure to DevOps practices and CI/CD pipelines for data engineering.
- Experience with other Big Data frameworks such as Spark Streaming or Kafka.
-
Big Data Architect
14 hours ago
Kannur, Kerala, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000Job Opportunity: Big Data ArchitectThe Senior Data Engineer position involves designing, building, and maintaining large-scale data architectures. This includes data lakes, lakehouses, and warehouses.Key Responsibilities:Create scalable real-time and batch processing workflows using PySpark and Apache Spark.Cleanse, transform, deduplicate, normalize, encode,...
-
Experienced Big Data Solutions Architect
2 days ago
Kannur, Kerala, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,20,00,000Our client seeks a highly skilled Senior Data Engineer to join their team. In this role, you will be responsible for designing, constructing, and optimizing large-scale data processing systems that enable advanced analytics and business intelligence.You will collaborate closely with data scientists, analysts, and other engineers to ensure the availability,...
-
Enterprise Data Engineer
16 hours ago
Kannur, Kerala, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000As a highly skilled data professional, you will play a key role in designing, building, and maintaining large-scale data processing systems. This includes architecting, developing, and deploying data pipelines that ensure high-performance, scalability, and reliability.The ideal candidate will have expertise in Azure-based technologies such as Data Factory...
-
High-Performance Data Engineer
2 days ago
Kannur, Kerala, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000Job Description:We're seeking a seasoned data engineer to design, develop and maintain high-performance data pipelines that process massive amounts of data. This role is critical in ensuring our data infrastructure can handle increasing volumes of data while maintaining exceptional performance standards.The ideal candidate will have experience with big data...
-
Strategic Data Infrastructure Specialist
2 days ago
Kannur, Kerala, India beBeeEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000Big Data Engineering Lead PositionWe are seeking a highly skilled and strategic leader with expertise in Big Data platforms, Microsoft Azure, and Databricks to spearhead the development of end-to-end data solutions.Key Responsibilities:Lead a team of experienced data engineers working on large-scale distributed data systems.Architect and oversee the design,...
-
Cloud Data Engineer
1 day ago
Kannur, Kerala, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000We're seeking a skilled Cloud Data Engineer to join our team. This is an exciting opportunity to work with cutting-edge technologies and build scalable, reliable, and cost-efficient cloud data platforms.The ideal candidate will have experience in designing and implementing data lake architectures on Databricks, as well as strong SQL and PL/SQL...
-
Data Science Lead
4 days ago
Kannur, Kerala, India beBeeData Full time ₹ 1,20,00,000 - ₹ 2,00,00,000Senior Data Engineering ProfessionalWe are seeking a highly skilled Senior Data Engineer to join our team in a mission-critical banking project. This is an exceptional opportunity to work on cutting-edge data pipelines, streaming platforms, and cloud-based infrastructures—building scalable, high-performance solutions for a leading financial services...
-
Data Engineer
3 weeks ago
Kannur, Kerala, India Garmin Technologies Full timeKey Responsibilities:Develop, maintain, and troubleshoot data pipelines using Apache Airflow, Python, and SQL/PLSQLManage and optimize data workflows for ETL, data cleaning, and transformation across multiple data sources and targetsCollaborate with the architect to evaluate design options and build scalable, high-quality solutionsPerform data administration...
-
Data Architect
14 hours ago
Kannur, Kerala, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000Big Data Architect RoleEffective data processing is a critical aspect of any organization. As such, we seek an experienced professional to fill the role of Big Data Architect.Key Responsibilities:Design and implement scalable data architectures using Apache Spark and related tools.Configure and maintain data pipelines for seamless data flow.Develop and...
-
Data Architect
1 day ago
Kannur, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 30,00,000Job RoleWe are seeking an experienced professional to design and build large-scale data infrastructures.The ideal candidate will have expertise in real-time data processing using technologies like Apache Kafka, Apache Flink, and Apache Spark. They will work closely with cross-functional teams to ensure seamless integration with existing systems and develop...