Databricks Engineer

3 weeks ago

west bengal, India Tredence Inc. Full time
Role: Senior Databricks Engineer / Databricks Technical Lead/ Data Architect
Experience: 4-12 years
Location: Kolkata
About Tredence:
Tredence focuses on last-mile delivery of powerful insights into profitable actions by uniting its strengths in business analytics, data science and software engineering. The largest companies across industries are engaging with us and deploying their prediction and optimization solutions at scale. Head quartered in the San Francisco Bay Area, we serve clients in the US, Canada, Europe, and South East Asia.
We are seeking an experienced data scientist who apart from the required mathematical and statistical expertise also possesses the natural curiosity and creative mind to ask questions, connect the dots, and uncover opportunities that lie hidden with the ultimate goal of realizing the data’s full potential.
Primary Roles and Responsibilities:
● Developing Modern Data Warehouse solutions using Databricks and AWS/ Azure Stack
● Ability to provide solutions that are forward-thinking in data engineering and analytics space
● Collaborate with DW/BI leads to understand new ETL pipeline development requirements.
● Triage issues to find gaps in existing pipelines and fix the issues
● Work with business to understand the need in reporting layer and develop data model to fulfill reporting needs
● Help joiner team members to resolve issues and technical challenges.
● Drive technical discussion with client architect and team members
● Orchestrate the data pipelines in scheduler via Airflow Skills and Qualifications:
● Bachelor's and/or master’s degree in computer science or equivalent experience.
● Must have total 6+ yrs. of IT experience and 3+ years' experience in Data warehouse/ETL projects.
● Deep understanding of Star and Snowflake dimensional modelling.
● Strong knowledge of Data Management principles
● Good understanding of Databricks Data & AI platform and Databricks Delta Lake Architecture
● Should have hands-on experience in SQL, Python and Spark (PySpark)
● Candidate must have experience in AWS/ Azure stack
● Desirable to have ETL with batch and streaming (Kinesis).
● Experience in building ETL / data warehouse transformation processes
● Experience with Apache Kafka for use with streaming data / event-based data
● Experience with other Open-Source big data products Hadoop (incl. Hive, Pig, Impala)
● Experience with Open Source non-relational / NoSQL data repositories (incl. MongoDB, Cassandra, Neo4J)
● Experience working with structured and unstructured data including imaging & geospatial data. ● Experience working in a Dev/Ops environment with tools such as Terraform, CircleCI, GIT.
● Proficiency in RDBMS, complex SQL, PL/SQL, Unix Shell Scripting, performance tuning and troubleshoot
● Databricks Certified Data Engineer Associate/Professional Certification (Desirable).
● Comfortable working in a dynamic, fast-paced, innovative environment with several ongoing concurrent projects
● Should have experience working in Agile methodology
● Strong verbal and written communication skills.
● Strong analytical and problem-solving skills with a high attention to detail.
Mandatory Skills: Python/ PySpark / Spark with Azure/ AWS Databricks
Tredence is an equal opportunity employer. We celebrate and support diversity and are committed to creating an inclusive environment for all employees. Visit our website for more details: Immediate to 30 Days

  • Azure data engineer

    2 months ago

    west bengal, India Tata Consultancy Services Full time

    Job Title- Databricks Developer Relevant Experience (in Yrs) 6 to 12 YRs Technical/Functi onal Skills · Minimum 5 years + working exp as Databricks Developer · Minimum 3 years + working exp on PySpark and AWS Experience Required 5 years+ Roles & Responsibilities · 5+ years relevant and progressive data engineering experience · Deep Technical knowledge...

  • Senior Data Engineer

    3 weeks ago

    west bengal, India Tredence Inc. Full time

    About Tredence: Tredence is a global data science solutions provider founded in 2013 by Shub Bhowmick, Sumit Mehra, and Shashank Dubey focused on solving the last-mile problem in AI. Headquartered in San Jose, California, the company embraces a vertical-first approach and an outcome-driven mindset to help clients win and accelerate value realization from...

  • Senior Data Engineer

    3 weeks ago

    west bengal, India Tredence Inc. Full time

    About Tredence:Tredence is a global data science solutions provider founded in 2013 by Shub Bhowmick, Sumit Mehra, and Shashank Dubey focused on solving the last-mile problem in AI. Headquartered in San Jose, California, the company embraces a vertical-first approach and an outcome-driven mindset to help clients win and accelerate value realization from...

  • Data engineer

    2 weeks ago

    West Bengal, India Webologix Ltd INC Full time

    Job Position: Azure Cloud DB Engineer Job Type: Full-Time Location: Kolkata, IN Experience: - 3 to 8+ Years Must Have: - Azure DATA Factory; Azure SQL; Azure Databricks; Azure Cosmos DB; Py Spark; Python Responsibilities:  Data Pipeline Development: Design and implement data integration workflows using Azure Data Factory to ingest,...

  • Azure data engineer

    3 weeks ago

    west bengal, India Webologix Ltd INC Full time

    Job Title: Azure Cloud Database Engineer Experience Required: 3-15 Years Location: Kolkata Job Type: Full-time Must Have: Azure DATA Factory; Azure SQL; Azure Databricks; Azure Cosmos DB; PySpark; Python; Snowflake Key Responsibilities: - Design, implement, and manage cloud-based OLTP databases using Azure SQL and Azure Cosmos DB. - Develop, optimize,...

  • Azure data engineer

    3 weeks ago

    west bengal, India Webologix Ltd INC Full time

    Job Title: Azure Cloud Database EngineerExperience Required: 3-15 YearsLocation: KolkataJob Type: Full-timeMust Have: Azure DATA Factory; Azure SQL; Azure Databricks; Azure Cosmos DB; PySpark; Python; SnowflakeKey Responsibilities:- Design, implement, and manage cloud-based OLTP databases using Azure SQL and Azure Cosmos DB.- Develop, optimize, and automate...

  • Azure data engineer

    3 weeks ago

    west bengal, India Webologix Ltd INC Full time

    Job Title: Azure Cloud Database EngineerExperience Required: 3-15 YearsLocation: KolkataJob Type: Full-timeMust Have: Azure DATA Factory; Azure SQL; Azure Databricks; Azure Cosmos DB; PySpark; Python; SnowflakeKey Responsibilities:Design, implement, and manage cloud-based OLTP databases using Azure SQL and Azure Cosmos DB .Develop, optimize, and...

  • Senior data engineer

    3 weeks ago

    Coimbatore West, India Deloitte Full time

    Hi, We are looking for Candidates who are having experience in Data Bricks Engineer Along with any cloud experience. Experience Required: 7- 14 Years Location: Coimbatore, Tamil Nadu, India Responsibilities Design, develop, and optimize data workflows and notebooks using Databricks to ingest, transform, and load data from various sources into the data lake....

  • Azure Databricks

    6 months ago

    Kolkata, West Bengal, India Cognizant Full time

    **Lead Administrator** **Qualification**: Bachelors in science, engineering or equivalent ** Responsibility**: Database/ tool Monitoring - Identify the specific parameters (Disk usage, performance metrics ) to be monitored and threshold values for those parameters. - Based on the criticality and understanding of the requirement, prioritize the specific...

  • Senior Data Engineer

    2 months ago

    Coimbatore West, India Deloitte Full time

    Hi, We are looking for Candidates who are having experience in Data Bricks Engineer Along with any cloud experience.Experience Required: 7- 14 YearsLocation: Coimbatore, Tamil Nadu, IndiaResponsibilitiesDesign, develop, and optimize data workflows and notebooks using Databricks to ingest, transform, and load data from various sources into the data lake.Build...

  • Pyspark/Python Data Engineer

    3 weeks ago

    west bengal, India Genpact Full time

    Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose – the relentless pursuit of a world that works better for people – we...