Big Data Engineering Leader

6 days ago

vadodara, India beBeeDataEngineer Full time

About the Position:We are seeking a seasoned Senior Data Engineer to lead our big data initiatives.Key Responsibilities:Design, build and deploy scalable, high-performance data pipelines using cloud-based tools like Azure Databricks, ADF and Spark.Create, optimize and maintain complex Spark (Scala/PySpark) jobs for large-volume data processing and transformation.Develop and manage data integration solutions from diverse sources including ADLS, Blob Storage, SQL databases, NoSQL stores and streaming platforms.Implement and support real-time data streaming using Kafka, Spark Streaming or Flink.Develop and maintain ETL workflows ensuring data quality, reliability and consistency.Work with cloud-based data warehousing and analytics solutions like Azure Synapse Analytics, Snowflake or BigQuery.Build and maintain CI/CD pipelines using DevOps tools for automated deployments.Collaborate with data architects, analysts and scientists to enable business insights through analytics and machine learning.Optimize query performance, storage, cost and compute usage across pipelines and environments.Ensure adherence to data governance, security, lineage and compliance best practices.Requirements:7–12 years of hands-on experience as a Data Engineer or Big Data Engineer.Strong expertise in cloud technologies, including Azure cloud services.Azure DatabricksAzure Data Factory (ADF)Azure Data Lake Storage (ADLS/Gen2)Azure Synapse/SQLExpert-level proficiency in Apache Spark, Spark SQL, PySpark or Scala.Experience with data streaming technologies like Kafka, Spark Streaming or Flink.Strong SQL skills with experience in performance tuning and query optimization.Hands-on experience with ETL tools like ADF, NiFi, Talend, Sqoop or Airflow.Good understanding of data modeling, warehouse concepts and distributed systems.Proficiency with Git, DevOps tools or similar CI/CD tools.Strong background in Linux/Unix, Bash or PowerShell scripting.Experience with cloud-based data warehousing solutions is a plus.Familiarity with NoSQL technologies like MongoDB, HBase, Cassandra or Cosmos DB.Preferred Qualifications:Experience in on-premise to cloud migration projects.Exposure to machine learning feature pipelines or MLOps workflows.Knowledge of data governance, security practices and metadata/catalog tools.Certifications in cloud technologies like Azure Data Engineer.

  • Big Data Migration Specialist

    4 days ago

    vadodara, India beBeeData Full time

    Big Data Migration Specialist Job Summary:We are seeking a skilled professional to spearhead the migration of reporting data sources from legacy systems to cloud-based BigQuery platforms.Migrate and validate existing Power BI reports to new BigQuery data modelsRepoint and optimize existing tables, views, and queries in BigQuery to support migrated...

  • Big Data Expert

    3 weeks ago

    Vadodara, India Namasys Analytics Full time

    Location: Muscat, OmanWorking Days: Sunday to ThursdayExperience: 6+ YearsNotice Period: Immediate joiners preferred (up to 30 days acceptable)SummaryAdminister and optimize enterprise big-data platforms with Cloudera, Spark, Kafka, Hive, Python, Hue, Cloud, API, and Rangers.ResponsibilitiesInstall, configure, upgrade, and operate Cloudera clusters; capacity...

  • Big data expert

    3 weeks ago

    Vadodara, India Namasys Analytics Full time

    Location: Muscat, Oman Working Days: Sunday to Thursday Experience: 6+ Years Notice Period: Immediate joiners preferred (up to 30 days acceptable) Summary Administer and optimize enterprise big-data platforms with Cloudera, Spark, Kafka, Hive, Python, Hue, Cloud, API, and Rangers. Responsibilities Install, configure, upgrade, and operate Cloudera clusters;...

  • Chief Data Platform Engineer

    4 days ago

    vadodara, India beBeeData Full time

    Big Data Architectural SpecialistWe are looking for an experienced Big Data Architectural Specialist with a proven track record in designing scalable data connectors. The ideal candidate will have expertise in software engineering, API integration and strong experience in building robust data platforms.Key Responsibilities:Design, develop and deploy...

  • Data Engineer

    7 days ago

    Vadodara, Gujarat, India Sharedpro Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Description :Roles and Responsibilities:Description of Skills/Needs :Duties and Responsibilities Design, build, and deliver efficient data models and products that align with business needs. Understand requirements fully and come up with robust data models and products while developing solutions tailored to analytical and consumption requirements....

  • Big Data Expertise Wanted

    2 days ago

    vadodara, India beBeeDataEngineerContractRole Full time

    Data Engineer - Contract RoleJob DescriptionWe are seeking a highly skilled Data Engineer to join our team in a contract role. As a Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems using PySpark.Key Responsibilities:Design and Develop Data Processing Systems: Use PySpark to design and...

  • Cloud Data Systems Engineer

    6 days ago

    vadodara, India beBeeDataEngineer Full time

    Key Responsibilities:Designing and implementing large-scale data processing systemsMaintaining complex data storage systems using cloud technologiesAnalyzing and visualizing big data sets with cloud-based toolsData engineers at a leading organization are responsible for designing, building, and maintaining large-scale data processing systems using Google...

  • Data Engineer-

    4 weeks ago

    Vadodara, India MyRemoteTeam Inc Full time

    About UsMyRemoteTeam, Inc is a fast-growing distributed workforce enabler, helping companies scale with top global talent. We empower businesses by providing world-class software engineers, operations support, and infrastructure to help them grow faster and better.Position: Python Coder – Data Processing Client: Wipro (Google Project) Location: Remote –...

  • Data Engineering Expertise Leader

    6 days ago

    vadodara, India beBeeData Full time

    Job Title: Principal Data EngineerWe seek a seasoned expert to spearhead the development and maintenance of our data infrastructure. The ideal candidate will possess at least 7-10 years of experience building and scaling large-scale data platforms.Key Responsibilities:Designing, implementing, and optimizing robust ETL/ELT pipelines for efficient data...

  • Senior Data Engineer

    19 hours ago

    Vadodara, India Ironbook AI Full time

    Role SummaryWe are looking for a Senior Data Engineer to design, build, and optimize high-performance data pipelines and data systems. The ideal candidate will have strong experience with cloud platforms, modern ETL/ELT tools, and deep technical skills in Python, SQL, and distributed data frameworks.Key Responsibilities- Design, develop, and maintain...