Big Data Specialist

1 week ago

Vadodara, Gujarat, India beBeeDataSpecialist Full time ₹ 6,00,000 - ₹ 12,00,000

We are seeking a seasoned Big Data Specialist to join our team. As a key player in our data engineering department, you will be responsible for designing and implementing large-scale data pipelines using modern tools and technologies.

Key Responsibilities

  • Design and Implement CI/CD Pipelines: Use GitHub Actions or equivalent tools to create and manage continuous integration and continuous deployment pipelines.
  • Maintain GitHub Branch Protection Rules: Configure and enforce branch protection rules and workflows to ensure smooth code integration and deployment.
  • Build and Optimize Spark Jobs: Develop and refine Spark jobs in Databricks for large-scale data processing and analysis.
  • Automate Databricks Deployments: Utilize Databricks Asset Bundles (DABs) or similar frameworks to automate deployments and reduce manual errors.
  • Develop Data Pipelines: Create robust data pipelines using Python and SQL, ensuring high-quality data and efficient processing.
  • Implement Unit Testing and Validation: Write unit tests and implement validation checks to ensure data workflows meet the required standards.
  • Manage Multi-Environment Deployments: Ensure seamless deployments across multiple environments, including Dev, Stage, and Prod, by maintaining consistency and reliability.
  • Troubleshoot Configuration Issues: Identify and resolve configuration or syntax issues in YAML and related files to minimize downtime.
  • Sustain Critical Notebooks: Maintain critical Databricks notebooks to ensure business continuity and optimal performance.
  • Migrate Legacy Reports: Transform legacy reports into modern, self-serve dashboards using Snowflake, Tableau, or Power BI, enhancing data accessibility and insights.
  • Collaborate with Cross-Functional Teams: Partner with cross-functional teams to understand needs and deliver actionable data models, insights, and reporting they can trust.

What We're Looking For:

  • Strong SQL, Databricks, and Python Skills: Proficiency in SQL, Databricks (Spark), and Python/R is essential for success in this role.
  • CI/CD Expertise: Experience with GitHub and CI/CD practices, including branch protection and GitHub Actions workflows, is crucial.
  • Unit Testing and Quality Checks: Strong understanding of unit testing and quality checks in data workflows is necessary for delivering high-quality results.
  • Databricks Asset Bundles: Familiarity with Databricks Asset Bundles (DABs) or similar automation tools is desired.
  • YAML Configuration Management: Knowledge of YAML configuration management, debugging, and error resolution is essential for success.
  • Problem-Solving and Debugging: Strong problem-solving and debugging skills are necessary for resolving complex data engineering workflow issues.
  • Snowflake or Cloud Data Warehouse: Experience with Snowflake or another cloud data warehouse is highly desirable.
  • Tableau/Power BI Dashboards: Hands-on experience with Tableau/Power BI dashboards and a good sense for usability is beneficial.

Why Join Us?

We offer a dynamic work environment that fosters innovation, collaboration, and growth. As a Big Data Specialist, you will have the opportunity to work on challenging projects, develop your skills, and make a meaningful impact on our organization.

  • Big Data Developer

    2 weeks ago

    Vadodara, Gujarat, India beBeeDataEngineering Full time ₹ 20,00,000 - ₹ 25,00,000

    Data Engineering SpecialistAbout the Role:Job Credentials: Experience - 5 to 12 years.Responsibilities:Skill Requirements: Big data spark, SQL, Hive, Scala.Our Expertise:Solutions: Data Aggregation & Visualization.Industries: Capital Markets, Asset & Wealth Management, Financial Services, Life Sciences, Manufacturing & Logistics, Software Product...

  • Big Data Innovator

    1 week ago

    Vadodara, Gujarat, India beBeeData Full time US$ 15,00,000 - US$ 20,00,000

    About This RoleWe're seeking a seasoned Big Data Engineer to drive the development of next-generation solutions across various sectors.Our ideal candidate has extensive experience in big data analytics platforms, Python/Java programming, CI/CD, infrastructure & cloud, and Terraform.Main Responsibilities:Develop and implement big data analytics solutions...

  • Lead Big Data Architect

    2 weeks ago

    Vadodara, Gujarat, India beBeeDataEngineer Full time ₹ 2,05,43,257 - ₹ 2,51,52,929

    Job Title: Lead Big Data ArchitectThis role involves designing and implementing highly scalable, distributed SaaS-based security risk prioritization products.The ideal candidate will have expertise in big data solutions using Spark and experience with engineering resources for innovation.They will be responsible for delivering roadmap features of Enterprise...

  • Big Data Administrator

    2 weeks ago

    Vadodara, Gujarat, India beBeebigdata Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    About UsWe are a leading organization in the field of big data and cloud technology. Our mission is to empower businesses to innovate and succeed through our expertise.

  • Senior Technical Specialist

    2 weeks ago

    Vadodara, Gujarat, India beBeeBigData Full time ₹ 25,00,000 - ₹ 30,00,000

    Job Opportunity: Big Data DeveloperWe are seeking a skilled Big Data Developer to join our delivery team. This role is ideal for a developer who thrives in a technical environment and is passionate about coaching and developing the next generation of talent.The successful candidate will work on live projects, provide technical support, contribute to solution...

  • Senior Big Data Data Scientist

    2 weeks ago

    Vadodara, Gujarat, India beBeeData Full time ₹ 15,63,179 - ₹ 21,14,286

    Job DescriptionWe are seeking an experienced data analytics professional with a strong background in Big Data technologies and Google Cloud Platform services to design and implement scalable data solutions.The ideal candidate will have extensive experience in designing, developing, and optimizing data pipelines using GCP BigQuery, Dataflow, and Apache...

  • Big Data Engineer

    1 week ago

    Vadodara, Gujarat, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Big Data EngineerThis job posting is for a highly motivated individual with expertise in PySpark, Python, SQL, and real-time data streaming.Role Overview:This is a contract remote role that requires building, optimizing, and maintaining large-scale data processing solutions. The ideal candidate will contribute to the design and implementation of...

  • Data Insights Specialist

    2 weeks ago

    Vadodara, Gujarat, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Opportunity: Data Insights SpecialistAbout the Role:We are seeking a highly skilled professional to drive business growth through data-driven insights. As a key member of our team, you will be responsible for handling and analyzing large-scale datasets to identify trends, patterns, and actionable business recommendations.Key Responsibilities:Work with...

  • Senior Big Data Engineer

    2 weeks ago

    Vadodara, Gujarat, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Description:We are seeking a highly skilled Senior Data Engineer to join our data engineering team. The ideal candidate will have deep expertise in building scalable data pipelines using Apache Spark, PySpark, SQL, and Python, along with hands-on experience in the AWS ecosystem.Key Responsibilities:Design and Develop Robust ETL/ELT PipelinesOptimize...

  • Data Insights Specialist

    2 weeks ago

    Vadodara, Gujarat, India beBeeBusiness Full time US$ 1,00,000 - US$ 1,20,000

    Job Title:Data Insights SpecialistResponsibilities:Analyze data from various sources to prepare comprehensive reports on customer behavior and campaign performance.Design and develop reports requested by stakeholders using business intelligence tools, conducting analyses of customer lifecycle activities.Produce dashboards measuring campaign effectiveness...