Senior Data Engineering Professional

7 days ago

Varanasi, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

Job Description

We are seeking an experienced Freelance Data Engineer to join our team in designing, building, and optimizing large-scale data pipelines using PySpark, AWS, Databricks, and Python/SQL.

Key Responsibilities:

  • Data Pipeline Development: Design, develop, and maintain ETL/ELT pipelines using PySpark, AWS, and Databricks for high-performance data integration and processing.
  • Data Profiling and Transformation: Perform data profiling, validation, and transformation using Python and SQL to ensure data quality and integrity.
  • Workflow Optimization: Optimize and monitor workflows for performance, scalability, and cost efficiency to minimize resource utilization.
  • Automation Scripting: Build automation scripts using Linux and Bash for data pipeline orchestration to improve productivity and reliability.
  • KPIs and Metrics: Collaborate with analytics teams to surface trustworthy KPIs and metrics to inform business decisions.
  • Troubleshooting: Troubleshoot and resolve data inconsistencies, configuration issues, and pipeline errors to ensure seamless data flow.
  • Cross-Functional Collaboration: Work closely with cross-functional teams (analysts, product managers, engineers) to define and implement data requirements for successful project delivery.
  • Documentation and Knowledge Sharing: Ensure thorough documentation and knowledge sharing across stakeholders to promote a culture of collaboration and innovation.

Required Skills and Qualifications

  • 5+ years of data engineering experience with a strong background in Python and SQL.
  • Proven expertise in PySpark, AWS services, and Databricks for large-scale data processing and integration.
  • Proficiency in Linux/Bash scripting for automation and data operations.
  • Hands-on experience with ETL/ELT pipelines and large-scale integrations for efficient data transfer.
  • Familiarity with Agile and CI/CD practices for rapid development and deployment.
  • Strong problem-solving and communication skills, with experience in client-facing roles for effective collaboration.

  • Advanced Data Engineering Professional

    1 day ago

    Varanasi, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 30,00,000

    Senior Data EngineerJob OverviewWe seek an accomplished senior data engineer to spearhead the development and maintenance of large-scale data engineering projects.The ideal candidate will possess 6+ years of experience in designing, developing, deploying, and optimizing complex data pipelines and ETL processes using Databricks.Key...

  • Senior Data Engineer

    4 days ago

    Varanasi, Uttar Pradesh, India beBeeData Full time ₹ 7,50,000 - ₹ 11,25,000

    Job Opportunity: Senior Data EngineerWe are seeking a highly skilled data engineer to join our team.Responsibilities:Design and implement efficient data pipelines using PythonCollaborate with cross-functional teams to develop and maintain large-scale data systemsWork closely with stakeholders to understand business needs and deliver high-quality...

  • Data Engineering Professional with Cloud Expertise

    18 hours ago

    Varanasi, Uttar Pradesh, India beBeeData Full time ₹ 18,00,000 - ₹ 24,00,000

    Senior Data EngineerGreetings to all interested data professionals!Role : We are seeking a Senior Data Engineer with expertise in Google Cloud Platform (GCP) and Extract, Transform, Load (ETL) processes.Location- This is a 100% remote position.Exp Range- Candidates with 4+ years of experience in data engineering are preferred.Notice Period- Immediate joiners...

  • Data Engineering Professional

    1 day ago

    Varanasi, Uttar Pradesh, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data Engineering ProfessionalJob OverviewAs a Data Engineering Professional, you will be responsible for designing and implementing large-scale data processing systems. Our team is looking for a highly skilled professional with 4+ years of hands-on experience in programming languages such as Java, Scala, and Unix Shell scripting.The ideal...

  • Career Advancing Senior Data Professional

    13 hours ago

    Varanasi, Uttar Pradesh, India beBeeMachineLearning Full time ₹ 1,50,00,000 - ₹ 2,01,00,000

    Unlock Career Advancement Opportunities">**Senior Data Scientist Job Description**At our organization, we're seeking a highly skilled data professional to drive strategic business outcomes. As a key member of the data science group, you'll be responsible for developing and deploying machine learning models, extracting insights from complex datasets, and...

  • Senior IoT Data Engineer

    1 week ago

    Varanasi, Uttar Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Senior IoT Data EngineerWe are seeking an experienced Senior IoT Data Engineer with hands-on coding expertise in Python and PySpark. The successful candidate will have a strong understanding of IoT systems, especially in refrigeration units, HVAC systems, controllers, and sensors.Key Responsibilities: Implement an application for a large retailer...

  • Senior Data Engineering Professional

    1 week ago

    Varanasi, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job SummaryWe are seeking a seasoned Data Engineer with extensive experience in Trino (PrestoSQL) and expertise in distributed computing frameworks and large-scale data systems. The ideal candidate will have a strong analytical mindset, excellent problem-solving skills, and the ability to work effectively in a collaborative environment.Key RequirementsA...

  • Chief Data Engineering Officer

    1 day ago

    Varanasi, Uttar Pradesh, India beBeeDataEngineering Full time ₹ 2,00,00,000 - ₹ 3,00,00,000

    Job Title: Senior Data Engineering LeadWe are seeking a highly experienced Senior Data Engineering Lead with a strong focus on Databricks and cloud-based data engineering to lead our data engineering team.This leadership role requires a visionary who can design, develop, and manage scalable data infrastructure and pipelines, while mentoring and inspiring a...

  • Highly Skilled Data Engineering Professional

    2 days ago

    Varanasi, Uttar Pradesh, India beBeeSenior Full time ₹ 1,50,00,000 - ₹ 2,25,00,000

    We are looking for a highly skilled professional to join our team as a Senior Data Engineer.About the RoleThe ideal candidate will be responsible for designing, developing and maintaining large-scale data processing systems that can handle high volumes of data.Key ResponsibilitiesDevelop efficient ETL pipelines as per business requirements following...

  • Senior Data Solutions Engineer

    19 hours ago

    Varanasi, Uttar Pradesh, India beBeeData Full time ₹ 14,00,000 - ₹ 22,40,000

    Job Title: Senior Data Solutions EngineerThis role is a pivotal opportunity for an experienced data engineer to join our team and contribute to the design, development, and implementation of cutting-edge data-centric solutions.We are looking for a highly skilled professional with expertise in building robust ETL pipelines using Spark and Scala programming...