Building Scalable Data Systems

1 day ago

Anantapur, Andhra Pradesh, India beBeeCloudDataEngineer Full time ₹ 80,00,000 - ₹ 1,20,00,000

Job Description

We are seeking a skilled Cloud Data Engineer to join our team. The ideal candidate will have experience in building data products using Databricks and related technologies.

The successful candidate will apply their skills to manage the existing cloud data platform, making it more scalable, reliable, and cost-efficient. They will also work on additional projects that leverage the existing architecture in some cases and use newer technologies where needed.

  • Analyze and understand existing data warehouse implementations to support migration and consolidation efforts.
  • Reverse-engineer legacy stored procedures (PL/SQL, SQL) and translate business logic into scalable Spark SQL code within Databricks notebooks.
  • Design and develop data lake solutions on AWS using S3 and Delta Lake architecture, leveraging Databricks for processing and transformation.
  • Build and maintain robust data pipelines using ETL tools with ingestion into S3 and processing in Databricks.
  • Collaborate with data architects to implement ingestion and transformation frameworks aligned with enterprise standards.
  • Evaluate and optimize data models (Star, Snowflake, Flattened) for performance and scalability in the new platform.
  • Document ETL processes, data flows, and transformation logic to ensure transparency and maintainability.
  • Perform foundational data administration tasks including job scheduling, error troubleshooting, performance tuning, and backup coordination.
  • Work closely with cross-functional teams to ensure smooth transition and integration of data sources into the unified platform.
  • Participate in Agile ceremonies and contribute to sprint planning, retrospectives, and backlog grooming.
  • Triage, debug, and fix technical issues related to Data Lakes.
  • Maintain and Manage Code repositories like Git.

Required Skills and Qualifications

  • Bachelor's degree in Computer Science, Information Technology, Data Engineering, or related field.
  • 5+ years of experience working with Databricks, including Spark SQL and Delta Lake implementations.
  • 3+ years of experience in designing and implementing data lake architectures on Databricks.
  • Strong SQL and PL/SQL skills with the ability to interpret and refactor legacy stored procedures.
  • Hands-on experience with data modeling and warehouse design principles.
  • Proficiency in at least one programming language (Python, Scala, Java).
  • Experience working in Agile environments and contributing to iterative development cycles.
  • Databricks cloud certification is a plus.
  • Exposure to enterprise data governance and metadata management practices.

  • Building Scalable Data Systems

    1 day ago

    Anantapur, Andhra Pradesh, India beBeeDatabase Full time ₹ 20,00,000 - ₹ 25,00,000

    PostgreSQL Database SpecialistAt the heart of our organization, we rely on experts like you to architect and implement cutting-edge data systems that drive growth and success.Design and deploy complex PostgreSQL databases using advanced query writing techniques, including CTEs, window functions, and indexing for optimal performance.Develop effective data...

  • Building Scalable Systems

    2 weeks ago

    Anantapur, Andhra Pradesh, India beBeeInfrastructure Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Reliable Infrastructure ExpertWe're seeking a highly skilled professional to join our team as a Reliable Infrastructure Expert. This role will play a critical part in designing, building, and maintaining the infrastructure that supports our global network.The ideal candidate will have a strong background in software development, with experience in...

  • Visionary Cloud Architect: Building Scalable Data Systems

    5 days ago

    Anantapur, Andhra Pradesh, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data Engineer Job DescriptionWe are seeking an experienced Senior Data Engineer to design and develop robust, scalable data pipelines that support analytics and reporting needs.The ideal candidate will have a strong background in cloud platforms, including Azure, Snowflake, and third-party technologies.They will collaborate with data scientists,...

  • Building Scalable AI Solutions

    3 days ago

    Anantapur, Andhra Pradesh, India beBeeArtificialIntelligence Full time ₹ 20,00,000 - ₹ 40,00,000

    AI Engineer WantedWe are seeking an experienced AI engineer to lead the development of our core intelligence behind a leading research platform.This role requires expertise in information retrieval, scalable backend systems, and generative AI.Design and deploy scalable APIs and services capable of serving hundreds of concurrent requests.Build and optimize...

  • Scalable Systems Engineer

    7 days ago

    Anantapur, Andhra Pradesh, India beBeeBackend Full time ₹ 10,00,000 - ₹ 16,00,000

    Backend Developer Job DescriptionWe are seeking an experienced Backend Developer to join our remote team in India. The successful candidate will be responsible for designing and building scalable backend systems that power our product, with a strong focus on data, cloud services, and reliability.The ideal candidate will have a deep understanding of...

  • Building High-Quality Data Systems

    3 days ago

    Anantapur, Andhra Pradesh, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    As a Data Engineering Lead, you will be responsible for developing and maintaining high-quality data ingestion and validation processes. You will design and implement scalable data quality validation systems across multiple sources, develop AI-driven tools to identify anomalies, and lead the development of data pipelines and validation scripts.Key...

  • Chief Architect of Scalable Systems

    3 days ago

    Anantapur, Andhra Pradesh, India beBeeBigData Full time ₹ 17,34,540 - ₹ 24,43,760

    Lead Big Data Engineer: Visionary RoleJob Description:We are seeking a Lead Big Data Engineer to deliver roadmap features of our Enterprise TruRisk Platform.As a key member of our team, you will be responsible for prototyping, designing, developing and supporting a highly scalable, distributed SaaS based Security Risk Prioritization product.This is an...

  • Building Scalable Ecommerce Solutions

    2 weeks ago

    Anantapur, Andhra Pradesh, India beBeeEcommerce Full time ₹ 80,00,000

    Shopify Developer PositionWe are seeking an experienced Shopify developer to join our team.The ideal candidate will have a proven track record of designing and building custom applications for e-commerce platforms.This is a full-time position that requires:Key Responsibilities:Designing and developing custom applications using Shopify LiquidIntegrating...

  • Develop Scalable Backend Systems

    1 week ago

    Anantapur, Andhra Pradesh, India beBeeBackendDeveloper Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    As a seasoned Backend Developer, you will play a pivotal role in designing and building scalable backend systems using Golang, Python, and JavaScript/TypeScript.This includes collaborating with cross-functional teams to create clean, efficient, and maintainable code that adheres to best practices. You will also be responsible for optimizing and...

  • Building Scalable Cloud Solutions

    5 days ago

    Anantapur, Andhra Pradesh, India beBeeCloud Full time US$ 15,00,000 - US$ 20,00,000

    As a Cloud Computing Strategist, you will be responsible for designing and implementing scalable, resilient, and cost-effective cloud architectures. You will lead infrastructure-as-code initiatives using Terraform, CloudFormation, and ARM templates, and architect and manage containerized environments with Docker and Kubernetes.Key Responsibilities:Design and...