
PySpark Developer
6 days ago
Job Description Key Skills & Responsibilities - Strong expertise in PySpark and Apache Spark for batch and real-time data processing. - Experience in designing and implementing ETL pipelines, including data ingestion, transformation, and validation. - Proficiency in Python for scripting, automation, and building reusable components. - Hands-on experience with scheduling tools like Airflow or Control-M to orchestrate workflows. - Familiarity with AWS ecosystem, especially S3 and related file system operations. - Strong understanding of Unix/Linux environments and Shell scripting. - Experience with Hadoop, Hive, and platforms like Cloudera or Hortonworks. - Ability to handle CDC (Change Data Capture) operations on large datasets. - Experience in performance tuning, optimizing Spark jobs, and troubleshooting. - Strong knowledge of data modeling, data validation, and writing unit test cases. - Exposure to real-time and batch integration with downstream/upstream systems. - Working knowledge of Jupyter Notebook, Zeppelin, or PyCharm for development and debugging. - Understanding of Agile methodologies, with experience in CI/CD tools (e.g., Jenkins, Git). Preferred Skills - Experience in building or integrating APIs for data provisioning. - Exposure to ETL or reporting tools such as Informatica, Tableau, Jasper, or QlikView. - Familiarity with AI/ML model development using PySpark in cloud environments - Skills: ci/cd,zeppelin,pycharm,pyspark,etl tools,control-m,unit test cases,tableau,performance tuning,jenkins,qlikview,informatica,jupyter notebook,api integration,unix/linux,git,aws s3,hive,cloudera,jasper,airflow,cdc,pyspark, apache spark, python, aws s3, airflow/control-m, sql, unix/linux, hive, hadoop, data modeling, and performance tuning,agile methodologies,aws,s3,data modeling,data validation,ai/ml model development,batch integration,apache spark,python,etl pipelines,shell scripting,hortonworks,real-time integration,hadoop. - Mandatory Key Skills - Apache Spark,Python,ETL,Unix,Linux,data engineering,Agile methodologies,CI/CD,data modeling,data validation,PySpark.
-
Python/Pyspark Developer
3 weeks ago
Gurugram, India DG Liger Consulting Full timeWe are looking for a skilled Python PySpark Developer with 34 years of experience in designing, developing, and maintaining big data solutions. The ideal candidate will have hands-on expertise in Python, PySpark, and data pipeline development, along with strong problem-solving skills and the ability to work in a collaborative environment.Key Responsibilities...
-
Lead Data Analyst
2 weeks ago
Gurugram, India Insight Global, LLC Full timeJob Description :Key Responsibilities :- Lead data-driven initiatives and mentor junior analysts within the analytics team.- Work cross-functionally with marketing, supply chain, and business units to deliver actionable insights.- Deploy analytics algorithms and tools on modern tech stacks for efficient data processing.- Develop data models, dashboards, and...
-
Pyspark +Python +SQL Professional
1 day ago
Bengaluru, Chennai, Gurugram, India Krazy Mantra HR Solutions Pvt. Ltd Full time ₹ 15,00,000 - ₹ 25,00,000 per yearWe are looking for a skilled professional with expertise in Pyspark, Python, and SQL to join our team. The ideal candidate should have 6-9 years of experience.Roles and ResponsibilityDesign, develop, and implement data processing pipelines using Pyspark.Collaborate with cross-functional teams to identify and prioritize project requirements.Develop and...
-
Python + Pyspark + SQL
3 weeks ago
Gurugram, India Talent Worx Full timeOur Client is a professional services firm, is the Indian member firm affiliated with International and was established in September 1993. Our professionals leverage the global network of firms, providing detailed knowledge of local laws, regulations, markets, and competition. Our client has offices across India in Ahmedabad, Bengaluru, Chandigarh,...
-
▷ (High Salary) Data Engineer(Pyspark+SQL)
2 weeks ago
Gurugram, Gurugram, India Accolite Full timeJob Description About The Role We are seeking an experienced Data Engineer to design, implement, and optimize a global data handling and synchronization solution across multiple regions. You will work with cloud-based databases, data lakes, and distributed systems, ensuring compliance with data residency and privacy requirements (e.g.,...
-
Azure Data Engineering Lead
3 weeks ago
Gurugram, India Techno Wise Full timeLocation : Gurugram (Work from Office 5 Days)Notice Period : Immediate to 45 DaysAbout the Role :We are looking for an experienced Azure Data Engineering Lead with strong expertise in Azure Data Factory, Databricks, PySpark, and SQL. This role requires both technical depth and leadership capability you will lead a team of 5+ engineers, architect and...
-
Bengaluru, Chennai, Gurugram, India Krazy Mantra HR Solutions Pvt. Ltd Full time ₹ 15,00,000 - ₹ 25,00,000 per yearWe are looking for a skilled professional with expertise in Databricks, Hadoop, Python, SQL, and Pyspark to join our team. The ideal candidate should have 6-9 years of experience.Roles and ResponsibilityDesign and develop scalable data pipelines using Databricks and Hadoop.Collaborate with cross-functional teams to integrate data from various sources.Develop...
-
Data Engineer, Pyspark
1 week ago
Gurugram, Gurugram, India NatWest Group Full timeJob Description Join us as a Data Engineer - We're looking for someone to build effortless, digital first customer experiences to help simplify our organisation and keep our data safe and secure - Day-to-day, you'll develop innovative, data-driven solutions through data pipelines, modelling and ETL design while inspiring to be commercially successful through...
-
ETL Developer
1 week ago
Gurugram, Gurugram, India Deqode Full timeJob Description Role - ETL Developer Work Mode - Hybrid Experience- 4+ years Location - Pune, Gurgaon, Bengaluru, Mumbai Required Skills Required Skills - AWS, AWS Glue, Pyspark, ETL, SQL - 4+ years of hands-on experience in MySQL, including SQL queries and procedure development - Experience in Pyspark, AWS, AWS Glue - Experience in AWS ,Migration -...
-
Software Engineer, PySpark And AWS
4 days ago
Bengaluru, Chennai, Gurugram, India Natwest Digitalx Full time ₹ 12,00,000 - ₹ 36,00,000 per yearJoin us as a Software Engineer, PySpark And AWSThis is an opportunity for a driven Software Engineer to take on an exciting new career challengeDay-to-day, you'll be engineering and maintaining innovative, customer centric, high performance, secure and robust solutionsIts a chance to hone your existing technical skills and advance your career while building...