Databricks Developer

2 days ago

Mumbai, Maharashtra, India Seaspan Corporation Full time

Position Description :

We are seeking a skilled Databricks Developer to join our Data Development team. Reporting to the Team Lead, Data Development, the Databricks Developer implements robust data pipelines using Apache Spark on Databricks, supports advanced data transformation, and enables scalable data products that serve enterprise analytics and reporting needs. Youll work closely with data engineers and analysts to ensure high-performance, reliable data pipelines and quality outputs.

This is a hands-on development role focused on engineering scalable, maintainable, and optimized data flows in a modern cloud-based environment.

Job Responsibilities :

- Design, build, and maintain scalable data pipelines and workflows using Databricks (SQL, PySpark, Delta Lake).

- Develop efficient ETL/ELT pipelines for structured and semi-structured data using Azure Data Factory (ADF) and Databricks notebooks/jobs.

- Integrate and transform large-scale datasets from multiple sources into unified, analytics-ready outputs.

- Optimize Spark jobs and manage Delta Lake performance using techniques such as partitioning, Z-ordering, broadcast joins, and caching.

- Design and implement data ingestion pipelines for RESTful APIs, transforming JSON responses into Spark tables.

- Apply best practices in data modeling and data warehousing concepts.

- Perform data validation and quality checks.

- Work with various data formats, including JSON, Parquet, and Avro.

- Build and manage data orchestration pipelines, including linked services and datasets for ADLS, Databricks, and SQL Server.

- Create parameterized and dynamic ADF pipelines, and trigger Databricks notebooks from ADF.

- Collaborate closely with Data Scientists, Data Analysts, Business Analysts, and Data Architects to deliver trusted, high-quality datasets.

- Contribute to data governance, metadata documentation, and ensure adherence to data quality standards.

- Use version control tools (e.g., Git) and CI/CD pipelines to manage code deployment and workflow changes.

- Develop real-time and batch processing pipelines for streaming data sources such as MQTT, Kafka, and Event Hub.

Requirements :

- 5+ years of experience in data engineering or big data development.

- Bachelor's degree in computer science or a relevant field, or equivalent training and work experience

- Strong hands-on experience with Databricks and Apache Spark (PySpark/SQL).

- Proven experience with Azure Data Factory, Azure Data Lake, and related Azure services.

- Experience integrating with APIs using libraries such as requests and http.

- Deep understanding of Delta Lake architecture, including performance tuning and advanced features.

- Proficiency in SQL and Python for data processing, transformation, and validation.

- Familiarity with data lakehouse architecture and both real-time and batch processing design patterns.

- Comfortable working with Git, DevOps pipelines, and Agile delivery methodologies

Additional Desired Qualifications :

- Experience with dbt, Azure Synapse, or Microsoft Fabric.

- Familiarity with Unity Catalog features in Databricks.

- Relevant certifications such as Azure Data Engineer, Databricks, or similar.

- Understanding of predictive modeling, anomaly detection, or machine learning, particularly with IoT datasets.

Job Demands and/or Physical Requirements :

As Seaspan is a global company, occasional work outside of regular office hours may be required.

(ref:hirist.tech)

  • Databricks Developer

    2 days ago

    Mumbai, Maharashtra, India Seaspan Corporation Full time US$ 1,20,000 - US$ 2,00,000 per year

    Seaspan teams are goal-driven and share a high-performance culture, focusing on building services offerings to become a leading asset manager. Seaspan provides many of the world's major shipping lines with alternatives to vessel ownership by offering long-term leases on large, modern containerships and pure car, truck carriers (PCTCs) combined with industry...

  • Azure Databricks Developer

    2 days ago

    Mumbai, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Data Engineer - Azure DatabricksOur organization seeks a skilled Data Engineer to lead the development of our data infrastructure using Azure Databricks and Pyspark.Key Responsibilities:Databricks Development:Design, develop, and deploy scalable and efficient data pipelines on Databricks.Data Engineering Expertise:Expert knowledge of Databricks, including...

  • Databricks

    1 week ago

    Mumbai, Maharashtra, India Talent Worx Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Every day, your work will make an impact that matters, while you thrive in a dynamic culture of inclusion, collaboration, and high performance. As the undisputed leader in professional services, Our Client is where you will find unrivaled opportunities to succeed and realize your full potential.We are seeking highly skilled Databricks Data Engineers to join...

  • High-Performance Databricks Developer

    2 weeks ago

    Mumbai, Maharashtra, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Contract Role: High-Performance Databricks DeveloperAbout the PositionWe are seeking an experienced Databricks developer to join our team on a short-term contract basis. As a key member of our engineering team, you will be responsible for designing and implementing high-performance data pipelines using Azure Data Factory.Key Responsibilities:Design and...

  • Databricks PySpark Developer Position

    6 days ago

    Mumbai, Maharashtra, India beBeeETL Full time ₹ 1,00,00,000 - ₹ 2,50,00,000

    Job Title: Data Engineer SpecialistWe are seeking a highly skilled Databricks PySpark Developer to join our team. As a key member of the data platform implementation team, you will be instrumental in designing, developing, and maintaining ETL processes to ensure efficient extraction, transformation, and loading of data from various sources into data lake and...

  • Senior Databricks Solutions Architect

    2 days ago

    Mumbai, Maharashtra, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    Job OverviewThis role focuses on leading and implementing data solutions using Databricks on the Azure cloud.The ideal candidate will have experience in designing, building, and optimizing Data Warehouses (DWH) and Data Lakes with scalable ETL applications. They should also be proficient in working with metadata-driven processing, CDC, and advanced...

  • Freelancer For Databricks With Spark

    3 days ago

    Mumbai, Maharashtra, India Mindmajic Technologies Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Company DescriptionMindmajic Technologies provides comprehensive cloud services for the design, development, integration, and maintenance of cloud-based apps, with expertise in AWS. We focus on personal interaction and mutual trust to achieve our goals, emphasizing cooperation and teamwork. We are committed to providing the best service and support to our...

  • Highly Skilled Data Engineer

    2 weeks ago

    Mumbai, Maharashtra, India beBeeDatabricks Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Databricks AdministratorAre you an experienced Databricks administrator looking for a new challenge? We are seeking a highly skilled professional to join our team as a Databricks Administrator.As a Databricks Administrator, you will be responsible for the day-to-day management of our Databricks platform, ensuring it runs smoothly and efficiently....

  • AWS Databricks

    3 days ago

    Mumbai, Maharashtra, India Future Focus Infotech Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Greetings from Future Focus InfotechWe have multiple opportunities AWS Databricks DeveloperExp: 5+yrsLocation : MumbaiJob Type- This is a Permanent position with Future Focus Infotech Pvt Ltd & you will be deputed with our client.A small glimpse about Future Focus Infotech Pvt Ltd. (Company URL: )If you are interested in above opportunity, send updated CV...

  • Senior Databricks Solutions Architect

    7 days ago

    Navi Mumbai, Maharashtra, India beBeeApplication Full time ₹ 1,04,000 - ₹ 1,30,878

    Job DescriptionWe are seeking an accomplished Application Developer to join our team. The successful candidate will be responsible for designing, building, and configuring applications that meet business process and application requirements.The ideal candidate will possess a deep understanding of Databricks Unified Data Analytics Platform and have experience...