Databricks Engineer- Technical lead

4 days ago

Pune, Maharashtra, India Symphoni Hr Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Locations:

Pune, Maharashtra, India

Minimum Experience:

8

Maximum Experience:

12

Skill to Evaluate:

azure/aws,databricks,SQL,PYSPARK, Team Handling

Job Title: Technical Lead

Company Name: NCSI

This role is accountable for running day to day operations of Data Platform in Azure / AWS Databricks NCS. Data Engineer is accountable for ongoing Development, Enhancement support and maintenance data availability and data quality, performance enhancement and stability of the system.

  1. Designing and implementing data ingestion pipelines from multiple sources using Azure Databricks

  2. Ensure data pipelines run smoothly and efficiently

  3. Adherence to security, regulatory and audit control guidelines

  4. Drive optimization, continuous improvement and efficiency

1

Designing and implementing data ingestion pipelines from multiple sources using Azure

Databricks

2 Ensure data pipelines run smoothly and efficiently

3

Developing scalable and re-usable frameworks for ingesting of data sets

4 Integrating the end to end data pipeline - to take data from source systems to target data

repositories ensuring the quality and consistency of data is maintained at all times

5 Working with event based / streaming technologies to ingest and process data

6 Working with other members of the project team to support delivery of additional project

components (API interfaces, Search)

7 Evaluating the performance and applicability of multiple tools against customer requirements

8 Technically competent in Cloud and Databricks to provide technical advice to the team and

to be involved in issue resolution

9 Provide on-call support and afterhours/weekend support as needed

10 Fulfill Service Requests related to Data Analytics platform

11 Lead and drive optimization and continuous improvement initiatives

12 Play gate-keeping role and conduct technical review of the changes as part of release

management

13 Understand various data security standards and adhere to the required data security controls

in the platform

14 Lead the design, development, and deployment of advanced data pipelines and analytical

workflows on the Databricks Lakehouse platform.

15 Collaborate with data scientists, engineers, and business stakeholders to build and scale

end-to-end data solutions.

16 Own architectural decisions and ensure adherence to data governance, security, and

compliance requirements.

17 Mentor a team of data engineers, providing technical guidance and career development.

Implement CI/CD practices for data engineering pipelines using tools like Azure DevOps,

GitHub Actions, or Jenkins.

Bachelors degree in IT, Computer

Science, Software Engineering,

Business Analytics or equivalent.

  • Relevant accredited

certifications for Azure, AWS,

Cloud Engineering and/or

Databricks

Work Experience

  • Minimum seven plus years of

experience in data analytics field

  • Experience with Azure/AWS

Databricks

  • Experience in building and optimizing

data pipelines, architectures and data

sets

  • Excellent experience in Scala or

Python, PySpark and SQL

  • Ability to troubleshoot and optimize

complex queries on the Spark

platform

  • Knowledgeable on structured and

unstructured data design / modelling,

data access and data storage

techniques

  • Expertise in designing and deploying

data applications on cloud solutions,

such as Azure or AWS

  • Hands on experience in performance

tuning and optimizing code running in

Databricks environment

  • Demonstrated analytical and

problem-solving skills particularly

those that apply to a big data

environment

  • Databricks Engineer

    3 days ago

    Pune, Maharashtra, India InfoCepts Full time ₹ 5,00,000 - ₹ 25,00,000 per year

    Position: Data Engineer- DatabricksPurpose of the Position:Develop, support and steer end-to-end business intelligence using Databricks.Location: Nagpur/ Pune/ Chennai/ BangaloreKey Responsibilities:Work with business analysts and data architects to translate requirements into technical implementations.Design, develop, implement, and maintain pyspark code...

  • Technical Lead

    1 week ago

    Pune, Maharashtra, India NCSI Technologies Full time ₹ 1,20,000 - ₹ 3,00,000 per year

    1 Designing and implementing data ingestion pipelines from multiple sources using Azure Databricks 2 Ensure data pipelines run smoothly and efficiently 3 Developing scalable and re-usable frameworks for ingesting of data sets 4 Integrating the end-to-end data pipeline - to take data from source systems to target data repositories ensuring the quality and...

  • Databricks Solution Architect

    2 weeks ago

    Pune, Maharashtra, India InspireXT Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    About InspireXTInspireXT is on a mission to build the most trusted supply chains in the world. Our Connected Intelligence vision combines Databricks with deep Oracle ERP/SCM, Salesforce, and industry accelerators to deliver compliant, AI-driven outcomes in Pharma, Manufacturing and Retail.We are investing significantly in our Databricks alliance and building...

  • Databricks Solution Architect

    1 week ago

    Pune, Maharashtra, India InspireXT Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Description :About InspireXT : InspireXT is on a mission to build the most trusted supply chains in the world. Our Connected Intelligence vision combines Databricks with deep Oracle ERP/SCM, Salesforce, and industry accelerators to deliver compliant, AI-driven outcomes in Pharma, Manufacturing and Retail. We are investing significantly in our...

  • Databricks Solution Architect

    3 days ago

    Pune, Maharashtra, India InspireXT Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    DescriptionAbout InspireXT :InspireXT is on a mission to build the most trusted supply chains in the world.Our Connected Intelligence vision combines Databricks with deep Oracle ERP/SCM, Salesforce, and industry accelerators to deliver compliant, AI-driven outcomes in Pharma, Manufacturing and Retail.We are investing significantly in our Databricks alliance...

  • Data Engineer- Databricks

    3 days ago

    Pune, Maharashtra, India InfoCepts Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Position: Data Engineer- DatabricksPurpose of the Position:Develop, support and steer end-to-end business intelligence using Databricks.Location: Nagpur/ Pune/ Chennai/ BangaloreKey Responsibilities:Work with business analysts and data architects to translate requirements into technical implementations.Design, develop, implement, and maintain pyspark code...

  • Databricks Engineer

    3 days ago

    Pune, Maharashtra, India Persistent Systems Full time ₹ 12,00,000 - ₹ 20,00,000 per year

    About Position:We are seeking data bricks engineer with hands on experience in Power BI, python, spark and data bricks.Role: Databricks EngineerLocation: Pune & MumbaiExperience: 8 to 12 YearsJob Type: Full Time EmploymentWhat You'll Do:3+ years of experience in data engineering or analytics roles.Strong hands-on experience with Databricks and Apache...

  • Databricks Engineer

    3 days ago

    Pune, Maharashtra, India Peoplefy Infosolutions Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Must have skills:Hands-on experience with Databricks platform, including Delta Lake/ Lakehouse architecturePerson should be familiar with Python / Pyspark code (not needed a developer)Knowledge of data structures, data modeling, ETL/ELT, and data warehousing.SQL basic in data querying and troubleshooting / manipulation.Key asks for the role:We need a...

  • Technical Lead

    1 week ago

    Pune, Maharashtra, India ITC Infotech Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Title: AWS Cloud Tech LeadExperience: yearsJob Description:React (FE)/ Typescript (BE) PostGreSQL are a must Fargate is a plusEveryone should be rest API thinking by default.Tech stack: AWS Cloudfront, S3 buckets, , typescript, , , , Aurora Postgres SQL, uses Rest APIs,uses Databricks as Data Lake, Auth0 for access/login Does this help? its microservice...

  • Databricks Developer

    5 days ago

    Pune, Maharashtra, India Luxoft Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Project descriptionYou will be working in a global team that manages and performs a global technical control. You'll be joining Assets Management team which is looking after asset management data foundation and operates a set of in-house developed tooling. As an IT engineer you'll play an important role in ensuring the development methodology is...