
Large Scale Data Processing Specialist
7 days ago
A Data Engineering Specialist with Expertise in PySpark
About the RoleWe are seeking an experienced Data Engineering Specialist to lead the design, development, and optimization of large-scale data processing solutions using PySpark.
- The ideal candidate will have hands-on expertise in PySpark, Python, SQL, and real-time data streaming, as well as experience working with Databricks in a production environment.
- They will be responsible for building, monitoring, and supporting real-time streaming solutions using Apache Kafka, and collaborating with cross-functional teams to deliver data-driven solutions.
Key Responsibilities:
- Design, develop, and optimize data pipelines using PySpark, Spark, and Python.
- Implement ETL/ELT solutions to integrate structured and unstructured data from multiple sources.
- Develop and maintain Hive queries and ensure seamless data transformations.
- Write and optimize SQL queries for large datasets.
- Collaborate with cross-functional teams to deliver data-driven solutions.
Required Skills & Qualifications:
- 3-4 years of experience in data engineering with a focus on PySpark and Python.
- Strong knowledge of Spark, Hive, SQL, and Unix shell scripting.
- Experience with Databricks in a production environment.
- Hands-on experience with Azure cloud services (preferred).
- Strong problem-solving and debugging skills.
Nice-to-Have Skills:
- Exposure to CI/CD pipelines for data solutions.
- Knowledge of data governance and security best practices.
-
Expertise in Large-Scale Data Processing
1 week ago
Anand, Gujarat, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000Senior Data Engineer Position OverviewThe role of Senior Data Engineer entails designing and implementing large-scale data processing systems.Lead the development of scalable data architectures and pipelines using AWS services such as EMR, Glue, Lambda, Step Functions, API Gateway, and Athena.Collaborate with cross-functional teams to deliver complex data...
-
Senior Data Science Engineer
1 week ago
Anand, Gujarat, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000Software Engineer At Incept Labs, we believe in harnessing the synergy between humans and AI to propel education and research forward. Our mission is to create a harmonious partnership where AI's processing capabilities are augmented by human imagination, values, and experience.Our team of scientists, engineers, and builders is dedicated to developing...
-
Large-Scale Database Expert
6 days ago
Anand, Gujarat, India beBeeDatabaseAdministrator Full time ₹ 15,00,000 - ₹ 20,00,000Job Title: Database AdministratorPostgreSQL DBA Job OpportunityWe are seeking an experienced PostgreSQL database administrator to join our team. The ideal candidate will have strong expertise in database administration, performance optimization, and query tuning for large-scale databases.Key Responsibilities:Administer, maintain, and optimize large-scale...
-
Freelance Big Data Specialist
2 weeks ago
Anand, Gujarat, India beBeeDataEngineer Full time ₹ 90,00,000 - ₹ 1,20,00,000We are seeking a seasoned Freelance Data Engineer to design and develop large-scale data pipelines using Python, SQL, and AWS.Key Responsibilities:Design, build, and maintain ETL/ELT pipelines using PySpark, AWS, and Databricks for robust and high-performance data solutions.Perform data profiling, validation, and transformation using Python and SQL.Optimize...
-
Anand, Gujarat, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000Job Opening:We are seeking a seasoned data engineer to spearhead the design, development, and maintenance of large-scale data processing systems on Google Cloud Platform (GCP).Key Responsibilities:ETL Process Development: Design and implement efficient ETL pipelines using Python or Java to extract data from various sources, transform it into a standardized...
-
Large Scale Event Manager
2 weeks ago
Anand, Gujarat, India beBeeEventProducer Full time ₹ 15,00,000 - ₹ 23,00,000Event Producer Role OverviewThis position involves managing the planning and execution of large-scale events to meet executive requirements.Key responsibilities include overseeing event strategy, identifying service gaps, and collaborating with stakeholders to align event production with corporate communications.Requirements:Exceptional organizational skills...
-
Chief Architect for Large-Scale Data Systems
5 days ago
Anand, Gujarat, India beBeeData Full time US$ 1,50,000 - US$ 2,50,000Senior Data EngineerThis role plays a key part in designing and developing large-scale data systems. As a bridge between engineering teams and business partners, you will contribute to the design and development of data pipelines, APIs, and platform components.Key Responsibilities:Drive development efforts by contributing high-quality, scalable, and...
-
Senior Data Engineer
4 days ago
Anand, Gujarat, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000Job Title: Data ArchitectWe are seeking a talented and experienced Data Architect to join our team. As a key member of our organization, you will be responsible for designing, building, and maintaining large-scale data systems.Key Responsibilities:Design and implement scalable data architectures that meet the needs of our business.Collaborate with...
-
Anand, Gujarat, India beBeeCreative Full time ₹ 10,00,000 - ₹ 15,00,000We're pushing the boundaries of creativity to unprecedented heights.Do you have what it takes to drive bold artistic visions and execute them with military precision?Job DescriptionSynthesize creative storytelling and operational excellence to deliver high-quality content at scale.As a seasoned Art Director, you'll be responsible for leading the creative...
-
Senior Operations Manager
2 weeks ago
Anand, Gujarat, India beBeeOperations Full time US$ 1,50,000 - US$ 2,00,000Job OverviewWe are seeking a seasoned Operations Manager with extensive experience in managing large-scale teams across Banking, Financial Services, and Insurance.The ideal candidate will bring 15+ years of expertise in setting up, transitioning, and managing operations teams, working with global clients across multiple regions.Key Responsibilities:Lead the...