Senior Data Engineer

3 weeks ago

kolkata, India FinArb Consulting Full time

Responsibilities

We are looking for a for a strong self-driven Senior Data Engineer to join the team of data and software ­­­engineering

 

Primary responsibilities would include: 

  

  • Investigate source data systems to build comprehensive data models to serve as Single Source of Truth
  • Design and implement flexible, robust & scalable ETL infrastructure
  • Deploy pipelines in higher environments after robust testing criteria
  • Create and maintain robust Big Data pipelines for Machine Learning systems
  • Optimize performance of Big Data scripts
  • Create monitoring and reporting infrastructure for Machine Learning systems
  • Develop systems that facilitate Data Science activities such as feature extraction and model testing
  • Design and build automated systems with pipelines, transformations, data quality analytics and control logic.
  • Generate, process, log and correlate events across multiple sources.


Qualifications

Basic qualifications:

 

  • Bachelor’s degree in Computer Science, Engineering, or closely related field
  • 2+ years of experience designing and developing software.
  • Proficient with Python, test-driven development, refactoring
  • Strong scripting skills to perform data/file manipulation
  • 1+ years of experience with data aggregation platforms based on technologies such as Spark, SQL, Azure Data Lake, etc.
  • Solid understanding and proven skills in raw and processed stream design, relational database design and dimensional models
  • Familiarity with machine learning and predictive analytics methods.
  • Experience using scalable data ingestion and transformation systems in batch and near real time environments.
  • Experience with SQL, relational modeling, and big data tools such as Cosmos, Hive, Scope or Spark
  • Experience with ETL, data modeling, and working with Business Intelligence systems.
  • Solid understanding of event processing including publish/subscribe mechanisms.
  • Demonstrated ability to create and ship high quality code by using engineering best practices.
  • Strong logical reasoning, problem solving and debugging skills
  • Experience with developing software in distributed, cloud-based systems
  • Experience with Big Data systems, Data Pipelines
  • Technical experience with Python, SQL, other scripting languages.
  • Keen interest in Machine Learning
  • A team player with strong communication skills and drive for results

 

Preferred qualifications:

 

  • Azure, AWS
  • Python
  • SQL
  • Knowledge in Machine Learning

  • Senior Data Engineer

    3 weeks ago

    Kolkata, India FinArb Consulting Full time

    ResponsibilitiesWe are looking for a for a strong self-driven Senior Data Engineer to join the team of data and software ­­­engineeringPrimary responsibilities would include:Investigate source data systems to build comprehensive data models to serve as Single Source of TruthDesign and implement flexible, robust & scalable ETL infrastructureDeploy...

  • Senior Data Engineer

    3 weeks ago

    Kolkata, India FinArb Consulting Full time

    ResponsibilitiesWe are looking for a for a strong self-driven Senior Data Engineer to join the team of data and software ­­­engineering Primary responsibilities would include:   Investigate source data systems to build comprehensive data models to serve as Single Source of TruthDesign and implement flexible, robust & scalable ETL infrastructureDeploy...

  • Senior Data Engineer

    3 weeks ago

    Greater Kolkata Area, India FinArb Consulting Full time

    ResponsibilitiesWe are looking for a for a strong self-driven Senior Data Engineer to join the team of data and software ­­­engineering Primary responsibilities would include:   Investigate source data systems to build comprehensive data models to serve as Single Source of TruthDesign and implement flexible, robust & scalable ETL infrastructureDeploy...

  • Senior Data Engineer

    4 weeks ago

    Kolkata, India Nilasu consulting Full time

    Summary :- We are seeking a highly skilled and experienced Senior Data Engineer to join our team.- You will play a key role in designing, developing, and maintaining large-scale data processing pipelines using Python and Spark/PySpark.- Your expertise in distributed computing frameworks and DevOps tools will be instrumental in building efficient and scalable...

  • [3 Days Left] Senior Data Engineer

    6 days ago

    Kolkata, India FinArb Consulting Full time

    Responsibilities We are looking for a for a strong self-driven Senior Data Engineer to join the team of data and software ­­­engineeringPrimary responsibilities would include:Investigate source data systems to build comprehensive data models to serve as Single Source of Truth Design and implement flexible, robust & scalable ETL infrastructure Deploy...

  • Specialist Data center operations

    4 weeks ago

    Kolkata, India NTT DATA Full time

    The role of Critical Facilities Maintenance Engineer is responsible for working with all major systems within a facility or building such as electrical and HVAC systems and performs regular maintenance on all systems. This position performs hands-on operations and maintenance which includes all physical and administrative operations tasks, service, and...

  • Specialist Data center operations

    4 weeks ago

    Kolkata, India NTT DATA Full time

    The role of Critical Facilities Maintenance Engineer is responsible for working with all major systems within a facility or building such as electrical and HVAC systems and performs regular maintenance on all systems. This position performs hands-on operations and maintenance which includes all physical and administrative operations tasks, service, and...

  • Receptionist/ Back-office Executive

    4 weeks ago

    Kolkata, India Glacier Engineer Full time

    **Responsibilities**: - Manage and maintain records and data in an organized and efficient manner - Process paperwork and documentation accurately and in a timely manner - Assist with scheduling appointments and managing calendars - Coordinate with other departments to ensure timely completion of tasks and projects - Perform general office duties such as...

  • Senior Machine Learning Engineer

    4 days ago

    Kolkata, India OptOut Full time

    Job Description :We are seeking an experienced Senior Machine Learning Engineer to join our team. The ideal candidate will have a strong background in developing production-grade machine learning models, as well as significant hands-on experience with ML technologies and techniques. The Senior Machine Learning Engineer will work closely with cross-functional...

  • Senior Machine Learning Engineer

    2 days ago

    kolkata, India OptOut Full time

    Job Description :We are seeking an experienced Senior Machine Learning Engineer to join our team. The ideal candidate will have a strong background in developing production-grade machine learning models, as well as significant hands-on experience with ML technologies and techniques. The Senior Machine Learning Engineer will work closely with cross-functional...

  • Senior Machine Learning Engineer

    4 days ago

    Kolkata, West Bengal, India OptOut Full time

    Job Description :We are seeking an experienced Senior Machine Learning Engineer to join our team. The ideal candidate will have a strong background in developing production-grade machine learning models, as well as significant hands-on experience with ML technologies and techniques. The Senior Machine Learning Engineer will work closely with cross-functional...

  • Senior Machine Learning Engineer

    4 days ago

    Kolkata, India OptOut Full time

    Job Description :We are seeking an experienced Senior Machine Learning Engineer to join our team. The ideal candidate will have a strong background in developing production-grade machine learning models, as well as significant hands-on experience with ML technologies and techniques. The Senior Machine Learning Engineer will work closely with cross-functional...

  • Senior Data Engineer

    4 weeks ago

    Bangalore,Kolkata,Pune, India Nilasu consulting Full time

    Summary :- We are seeking a highly skilled and experienced Senior Data Engineer to join our team.- You will play a key role in designing, developing, and maintaining large-scale data processing pipelines using Python and Spark/PySpark.- Your expertise in distributed computing frameworks and DevOps tools will be instrumental in building efficient and scalable...

  • Senior Data Engineer

    4 weeks ago

    Bangalore/Kolkata/Pune, IN Nilasu consulting Full time

    Summary :- We are seeking a highly skilled and experienced Senior Data Engineer to join our team.- You will play a key role in designing, developing, and maintaining large-scale data processing pipelines using Python and Spark/PySpark.- Your expertise in distributed computing frameworks and DevOps tools will be instrumental in building efficient and scalable...

  • Data Engineer

    3 days ago

    kolkata, India HTD Resources Full time

    Skill: Data Engineer with Python knowledge Designation : Software Engineer Relevant Experience: 5 - 6 years Mandatory Skills: Primary Skills - Python, SQL, AWS, DWH Concept, ELT, Secondary Skills - GCP, CDC Location: Mumbai / Kolkata Mode: Work from Office Interview Rounds: 3-4 Rounds

  • Software Engineer-Data Engineer

    4 weeks ago

    Kolkata, India CliniSys Full time

    Essential FunctionsExtract data from on-prem databases using appropriate tools and techniques.Transform and clean the data to ensure its suitability for PowerBI analysis.Build and maintain data pipelines for regular data refresh in PowerBI.Monitor and maintain Azure Synapse pipelines for on-prem data extraction and transformation.Collaborate with analysts to...

  • Senior DevOps Engineer

    3 weeks ago

    Kolkata, India Indus Net Technologies Ltd Full time

    We are looking for a Senior DevOps Engineer to manage the software development process and create an automated delivery pipeline that assists in building software more efficiently. The responsibilities of a DevOps Engineer include assessing, supporting, and implementing high-quality information technology architecture.Responsibilities:1. Build & Release...

  • Senior Principal Consultant

    2 weeks ago

    Kolkata, India Genpact Full time

    A Data Architectwho can design and implement data modernization solutions in the cloud is responsible for developing and implementing data architecture strategies for organizations transitioning their data systems to the cloud. They collaborate with stakeholders to understand business requirements and translate them into scalable, secure, and high-performing...

  • Senior Project Engineer

    1 month ago

    Kolkata, West Bengal, India ABB Full time

    Senior Project Engineer - Metals ProcessTake your next career step at ABB with a global team that is energizing the transformation of society and industry to achieve a more productive, sustainable future.At ABB, we have the clear goal of driving diversity and inclusion across all dimensions: gender, LGBTQ+, abilities, ethnicity and generations. Together, we...

  • Senior Composite Manufacturing Engineer

    2 days ago

    Greater Kolkata Area, India Drones Tech Lab Full time

    Company Description Drones Tech Lab is a company located in the Greater Kolkata Area that specializes in providing customized geo-spatial solutions through unmanned systems. With a focus on speed, accuracy, innovation, and a resilient service ethic, our solutions include hardware, software, and data analysis systems for surveillance purposes. Our mission is...