PySpark Developer
7 days ago
This is a walk-in drive on Mar 22 (Saturday) at Cochin/ Pune / Chennai location.
If you are interested in attending this walk-drive please share your updated resume to [Confidential Information]
- Mandatory: Primary skill: Pyspark
- Education: B. Tech/M. Tech/MCA/MS/MBA
- 5-12 years of experience in the design and implementation of Big Data systems using PySpark, database migration, transformation and integration solutions for any Data warehousing project.
- Must have excellent knowledge in Apache Spark and Python programming experience
- Deep experience in developing data processing tasks using PySpark such as reading data from external sources, merging data, performing data enrichment and loading into target data destinations.
- Experience in deployment and operationalizing the code, knowledge of scheduling tools like Airflow, Control-M etc. is preferred
- Working experience on Cloud technology architecture like AWS ecosystem, Google Cloud, BigQuery etc. is an added advantage
- Understanding of Unix/Linux + Shell Scripting
- Data modeling experience using advanced statistical analysis, unstructured data processing
- Experience with building APIs for provisioning data to downstream systems by leveraging different frameworks.
- Hands-on project experience on Jupyter Notebook/ Zeppelin/ PyCharm etc. IDEs
- Hands-on experience with AWS S3 Filesystem operations
- Good knowledge of Hadoop, Hive, and Cloudera/ Hortonworks Data Platform
- Experience handling CDC operations for huge volumes of data
- Should understand and have operating experience with Agile delivery methodologies
- Should have hands-on experience in the following: data validation, writing unit test cases
- Should have experience in integrating PySpark with downstream and upstream applications through a batch/real-time interface
- Should have experience in fine-tuning process and troubleshooting performance issues
- Should have demonstrated expertise in the development of design documents like HLD, LLD etc.
- Should have experience in leading requirements gathering and developing solution architecture for Data migration/integration initiatives
- Should have experience in handling client interactions at different phases of the projects
- Should have experience in leading a team in a project or a module
- Should be well versed with onsite/offshore model and its challenges
Skills: agile methodologies,data validation,airflow,cloud,aws s3,bigquery,zeppelin,cloud technology (aws ecosystem, google cloud, bigquery),agile delivery methodologies,hadoop,hive,shell scripting,hortonworks data platform,control-m,data modeling,aws,jupyter notebook,cdc operations,hortonworks,python,pycharm,google cloud,cloudera,api development,pyspark,apis,unit testing,data warehousing,apache spark,unix/linux
-
Pyspark Developer
7 days ago
Chennai, Tamil Nadu, India MP DOMINIC AND CO Full timeJob Summary - Design develop and implement scalable data pipelines and streaming use cases using PySpark and Spark on a distributed computing platform - Possess strong programming skills in Spark streaming - Have familiarity with cloud platforms like GCP - Gain experience in big data technologies such as Hadoop Hive and HDFS - Perform ETL operations...
-
PySpark Developers
6 hours ago
Chennai, Tamil Nadu, India LTIMindtree Full timeSkill: PySpark Developer/ConsultantJob Locations: Chennai, PuneNotice Period: AnyExperience: 3-12 yearsJob Description:PySpark DeveloperMandatory Skills: (Apache Spark, Big Data Hadoop Ecosystem, SparkSQL, Python)A good professional experience in Bigdata PySpark HIVE Hadoop PLSQLGood knowledge of AWS and SnowflakeGood understanding of CICD and system...
-
Pyspark Developer
3 weeks ago
Chennai, Tamil Nadu, India CIEL HR Full timeJob Title Pyspark DeveloperJob Location ChennaiJob Type C2HWork from office5 days workingExperience 3 to 8 yearsSkills 3 years Pyspark Developer with HadoopAbout CIEL HR Ciel HR marks the return of HR visionaries to the HR Services industry powered by technology and analytics The Founders of Ma Foi the pioneer of the HR services sector in India ...
-
Senior PySpark Developer
7 days ago
Chennai, Tamil Nadu, India Cognisol Full timeCognisol seeks a skilled Senior PySpark Developer to join its team. As a key member of our data engineering group, you will design and implement big data systems using PySpark, ensuring seamless data migration, transformation, and integration for our data warehousing projects.We are looking for an expert in Apache Spark and Python programming who can develop...
-
Pyspark Developer
2 weeks ago
Chennai, Tamil Nadu, India Newt Global Full timeDesignation Big Data Engineer Primary skills Pyspark Python Spark HDFS Hive Scala Location Chennai Gurgaon WFO Experience 5 - 7 Years Notice Period Immediate to 30 Days Serving Notice Desired Profiles Subject matter expert in Python and Spark 3 Years of experience in Hadoop Spark Scala PySpark Hive Impala and SQL Technologies Analyzed...
-
PySpark Developer Lead
48 minutes ago
Chennai, Tamil Nadu, India O2F Info Solutions Pvt. Ltd. Full timeRequired Skills and Qualifications:Experience: 4-8 years of experience in data engineering, with a strong focus on PySpark and large-scale data processing.Technical Skills: Expertise in PySpark for distributed data processing, data transformation, and job optimization. Strong proficiency in Python and SQL for data manipulation and pipeline creation.Hands-on...
-
PySpark Engineer
7 days ago
Chennai, Tamil Nadu, India MP DOMINIC AND CO Full timeAt MP DOMINIC AND CO, we are seeking a highly skilled PySpark Engineer to join our team. In this role, you will be responsible for designing and developing scalable data pipelines and streaming use cases using PySpark and Spark on a distributed computing platform.The ideal candidate will possess strong programming skills in Spark streaming, have familiarity...
-
PySpark Architect
6 days ago
Chennai, Tamil Nadu, India Cognisol Full timeCognisol is seeking a talented PySpark Architect to drive the design and implementation of our big data systems. As a key member of our data engineering team, you will use your expertise in PySpark, Apache Spark, and Python programming to develop efficient data processing tasks and ensure seamless data migration, transformation, and integration for our data...
-
Pyspark Architect
1 week ago
Chennai, Tamil Nadu, India Saaki Argus & Averil Consulting Full timeAbout Company: A global IT services and consulting company specializing in digital transformation, cloud services, and automation solutions. Role: Pyspark Architect Experience: 15 -22 Years Location: Mumbai, Pune, Chennai, Bangalore & Noida. Job Description:Proven experience as a Technical Architect or similar role with a strong focus on Python...
-
Pyspark Data Engineer
2 weeks ago
Chennai, Tamil Nadu, India Synechron Full timeAbout the Role: We are seeking a highly skilled Data Engineer with deep expertise in PySpark and the Cloudera Data Platform (CDP) to join our data engineering team. As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines that ensure high data quality and availability across the organization. This role...
-
Pyspark Data Engineer
3 weeks ago
Chennai, Tamil Nadu, India Synechron Full timeAbout the Role:We are seeking a highly skilled Data Engineer with deep expertise in PySpark and the Cloudera Data Platform (CDP) to join our data engineering team. As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines that ensure high data quality and availability across the organization. This role...
-
PySpark Developer
6 days ago
Chennai, Tamil Nadu, India LTIMindtree Full timeAbout UsLTIMindtree is a global IT services company that offers cutting-edge technology solutions to help businesses succeed. Our team of experts delivers high-quality technology solutions that drive business growth and improve efficiency.Job SummaryWe are seeking a skilled Bigdata Technologist to join our team in Chennai or Pune. As a Bigdata Technologist,...
-
Azure Cloud Engineer
6 days ago
Chennai, Tamil Nadu, India Tata Consultancy Services Full timeJob Description:Tata Consultancy Services is seeking a skilled Azure Cloud Engineer with experience in Pyspark development to join our team.About Us:We are a leading IT services company that provides innovative solutions to clients across various industries. Our team works collaboratively to deliver high-quality projects on time and within budget.Key...
-
PySpark Application Developer
3 weeks ago
Chennai, Tamil Nadu, India Corporate Resources Full timeJob Description Work on client projects to deliver AWS, PySpark, Databricks based Data engineering Analytics solutions Build and operate very large data warehouses or data lakes ETL optimization, designing, coding, tuning big data processes using Apache Spark Build data pipelines applications to stream...
-
Big Data Engineer
7 days ago
Chennai, Tamil Nadu, India Cognisol Full timeWe are searching for a highly skilled Big Data Engineer - PySpark to lead our data migration/integration initiatives. The successful candidate will possess excellent knowledge of PySpark, Apache Spark, and Python programming, with a strong focus on designing and implementing big data systems.This role involves collaborating with cross-functional teams to...
-
SQL Developer PySpark Analytics
6 hours ago
Chennai, Tamil Nadu, India 3Analytics Full timeAbout This Role We are seeking a skilled Data Architect to join our team at 3Analytics. As a key member of our data engineering team, you will be responsible for designing and implementing efficient data pipelines using SQL and Python. Your expertise in big data technologies such as Snowflake and AWS will enable us to unlock new insights and drive business...
-
io - Lead Data Engineer - ETL/PySpark
1 week ago
Chennai, Tamil Nadu, India Recro Full timeJob Description :We are looking for an experienced Lead Data Engineer to join our dynamic team and help us build innovative data engineering solutions that empower businesses to leverage the full potential of their data. As a Lead Data Engineer, you will be responsible for building scalable data pipelines, managing large datasets, and designing end-to-end...
-
Snowflake DB Developer
3 weeks ago
Chennai, Tamil Nadu, India Sentient Scripts Private Limited Full timeJob DescriptionDescription: We are seeking a skilled database developer with expertise in Snowflake, ADF, Pyspark, and Databricks to join our team. The ideal candidate will have a strong background in database development and data engineering.Responsibilities:- Design, develop, and maintain database solutions using Snowflake, ADF, Pyspark, and Databricks- -...
-
Database Developer Expert
5 hours ago
Chennai, Tamil Nadu, India Sentient Scripts Private Limited Full timeWe are seeking a skilled database developer with expertise in Snowflake, ADF, Pyspark, and Databricks to drive our data engineering efforts. The ideal candidate will have a strong background in database development and design.Key ResponsibilitiesDesign, develop, and maintain database solutions using Snowflake, ADF, Pyspark, and DatabricksOptimize database...
-
Development
3 weeks ago
Chennai, Tamil Nadu, India HCLTech Full timeHCL Technologies is HiringLocation: Pan IndiaExperience: 4+ yearsWe are looking for a Power BI Developer with strong data modeling and cloud ETL expertise to join our growing team at HCL Technologies If you're passionate about data analytics and cloud technologies, this could be the perfect opportunity for you.Key Responsibilities:Design and implement ...