Apache Spark Expert
6 days ago
Position Title:
- Apache Spark Expert**Location**:
- Pune, MH, OnsitePosition Type:
- Full-TimeInterview Process:
- Webcam/SkypeReq Id:
- 3314Salary/Rate:
- MarketPosition Duration:
- Permanent**Key Responsibilities**:
- Provide architectural consultancy to clients and develop implementation roadmaps for Apache Spark and Kafka solutions.
- Translate architectural strategies into code and facilitate knowledge transfer to junior developers.
- Demonstrate in-depth knowledge of AWS services, particularly MSK (Managed Streaming for Kafka), Spark EMR, and working with various data sources, sinks, and transformations.
- Proficiency in utilizing S3, Athena, Glue, and Step Functions for efficient data processing and analysis.
- Experience with Kinesis, Firehose, and the Kafka Connect Framework, including the development of custom connectors.
**Qualifications and Skills**:
- Strong expertise in Apache Spark and Kafka with a focus on architectural understanding.
- Proven hands-on experience in developing and implementing solutions using Apache Spark and Kafka Streaming.
- Sound knowledge of AWS services, including MSK, Spark EMR, S3, Athena, Glue, Step Functions, Kinesis, and Firehose.
- Proficiency in the Kafka Connect Framework, especially in the development of custom connectors.
**Preferred**:
- Familiarity with Apache Airflow is considered a plus.
- Experience in (Real-time) Data analytics is also welcome.
-
Apache Iceberg
1 week ago
Hyderabad, Pune, Bhubaneswar, India Tata Consultancy Services Limited Full timeJob Description Must Have Skills: - Good experience in Apache Iceberg, Apache Spark, Trino - Proficiency in SQL and data modeling Nice To Have Skills: Technical/Functional Skills Key Qualifications - Experience with open Data Lakehouse using Apache Iceberg - Experience with Data Lakehouse architecture with Apache Iceberg and Trino - Experience with...
-
Apache Spark Developer
1 day ago
Pune, India People Prime Worldwide Full timeAbout Client :- Our client is a French multinational information technology (IT) services and consulting company, headquartered in Paris, France. Founded in 1967, It has been a leader in business transformation for over 50 years, leveraging technology to address a wide range of business needs, from strategy and design to managing operations. The company is...
-
Hinduja Tech
3 weeks ago
Pune, India HINDUJA TECH LIMITED Full timeDescription :Data Engineer will be responsible for development and deployment of end-to-end data ingestion pipelines using Azure Databricks, Apache Spark, and related technologies, ensuring scalability, performance, and efficiency.Key responsibilities and tasks :- Design & Architecture : Design high-performance, resilient, and scalable data architectures for...
-
Databricks, Java Spark
1 week ago
Pune, India Bohiyaanam Technologies Pvt Ltd Full time**Position**:Databricks Java Spark **Location: Pune/Bangalore/Mumbai** **Experience: 6 to 9 Years** **Skills required**: Big Data-Apache Spark-Java **Responsibilities**: - **Primary skillset: Databricks, Spark,**: - **Exp - 5+ yrs**: - Spark, Scala or PySpark pipelines development in multihop pipeline architecture - Strong understanding of full CI/CD...
-
Tcs data Streaming
4 days ago
Pashan, Pune, Maharashtra, India Response Informatics Full timeJOB DESCRIPTION: Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster. Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data. SKILLS: Pyspark Scala (with a focus on the functional programming paradigm) Apache Spark 2.x, 3.x -...
-
Spark Admin
2 weeks ago
Pune, Maharashtra, India Virtusa Full timeSpark Administration: Install, configure, and manage Apache Spark 3.x. clusters. Monitor and optimize Spark cluster performance. Troubleshoot and resolve Spark-related issues. Manage Spark job scheduling and resource allocation. Ensure high availability and reliability of Spark clusters. DevOps Practices: Implement and manage CI/CD pipelines using tools like...
-
Apache Flink Engineer
4 days ago
Pune, Maharashtra, India Systems Plus Full time ₹ 2,40,001 - ₹ 9,00,001 per yearLocation: PuneEmployment Type: full-timeJob Detailsob Description: Apache Flink Engineer (minimum 4+ Years Experience)Position: Apache Flink EngineerExperience: 4+ years (Min. 2 years in Apache Flink)Location: RemoteType: Full-time/ContractJob RoleWe are looking for an experienced Data Engineer with strong hands-on expertise in Apache Flink, large-scale data...
-
Spark Developer
2 weeks ago
Bengaluru, Hyderabad, Pune, India Infosys Full time ₹ 12,00,000 - ₹ 36,00,000 per yearWe are seeking an experienced Spark Developer with 5+ years of expertise in big data processing and distributed computing. The ideal candidate will design, develop, and optimize data pipelines using Apache Spark and related technologies, ensuring high performance and scalability for large datasets.Key ResponsibilitiesDesign and implement Spark-based ETL...
-
Senior Spark Developer
5 days ago
Pune, India Ixceed Solutions Full timeDescription :Job Description :We are looking for an experienced Senior Spark Developer with a strong background in Scala, Java, and Big Data technologies. The ideal candidate will have 5+ years of development experience, including expertise in Spark-based data processing, Java server-side development, and working in fast-paced, low-latency environments. A...
-
Software Engineer
4 weeks ago
Pune, India Alp Consulting Ltd. Full timeJob Description: 5.5- 7.5 Years of in Big Data & Data related technology experience Expert level understanding of distributed computing principles Expert level knowledge and experience in Apache Spark Hands on programming with Python Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop Experience with building stream-processing systems, using technologies...