Senior Data Engineer

3 weeks ago


Greater Kolkata Area, India FinArb Consulting Full time

Responsibilities

We are looking for a for a strong self-driven Senior Data Engineer to join the team of data and software ­­­engineering

 

Primary responsibilities would include: 

  

  • Investigate source data systems to build comprehensive data models to serve as Single Source of Truth
  • Design and implement flexible, robust & scalable ETL infrastructure
  • Deploy pipelines in higher environments after robust testing criteria
  • Create and maintain robust Big Data pipelines for Machine Learning systems
  • Optimize performance of Big Data scripts
  • Create monitoring and reporting infrastructure for Machine Learning systems
  • Develop systems that facilitate Data Science activities such as feature extraction and model testing
  • Design and build automated systems with pipelines, transformations, data quality analytics and control logic.
  • Generate, process, log and correlate events across multiple sources.


Qualifications

Basic qualifications:

 

  • Bachelor’s degree in Computer Science, Engineering, or closely related field
  • 2+ years of experience designing and developing software.
  • Proficient with Python, test-driven development, refactoring
  • Strong scripting skills to perform data/file manipulation
  • 1+ years of experience with data aggregation platforms based on technologies such as Spark, SQL, Azure Data Lake, etc.
  • Solid understanding and proven skills in raw and processed stream design, relational database design and dimensional models
  • Familiarity with machine learning and predictive analytics methods.
  • Experience using scalable data ingestion and transformation systems in batch and near real time environments.
  • Experience with SQL, relational modeling, and big data tools such as Cosmos, Hive, Scope or Spark
  • Experience with ETL, data modeling, and working with Business Intelligence systems.
  • Solid understanding of event processing including publish/subscribe mechanisms.
  • Demonstrated ability to create and ship high quality code by using engineering best practices.
  • Strong logical reasoning, problem solving and debugging skills
  • Experience with developing software in distributed, cloud-based systems
  • Experience with Big Data systems, Data Pipelines
  • Technical experience with Python, SQL, other scripting languages.
  • Keen interest in Machine Learning
  • A team player with strong communication skills and drive for results

 

Preferred qualifications:

 

  • Azure, AWS
  • Python
  • SQL
  • Knowledge in Machine Learning


  • Greater Kolkata Area, India Drones Tech Lab Full time

    Company Description Drones Tech Lab is a company located in the Greater Kolkata Area that specializes in providing customized geo-spatial solutions through unmanned systems. With a focus on speed, accuracy, innovation, and a resilient service ethic, our solutions include hardware, software, and data analysis systems for surveillance purposes. Our mission is...

  • Data Engineer

    15 minutes ago


    Greater Bengaluru Area, India Genpact Full time

    Inviting applications for the role of Principal Consultant-Senior Data EngineerInclude optimizing data pipelines, ensuring data integrity and consistency, enhancing system resiliency where applicable, maintaining and improving data security, proactive alerting, and monitoring for data pipelines, and automating repetitive data-oriented...

  • Senior Data Engineer

    3 weeks ago


    Kolkata, India FinArb Consulting Full time

    ResponsibilitiesWe are looking for a for a strong self-driven Senior Data Engineer to join the team of data and software ­­­engineeringPrimary responsibilities would include:Investigate source data systems to build comprehensive data models to serve as Single Source of TruthDesign and implement flexible, robust & scalable ETL infrastructureDeploy...

  • GCP Data Engineer

    1 month ago


    Greater Bengaluru Area, India Tredence Inc. Full time

    Have Implemented and Architected solutions on Google Cloud Platform using the components of GCP Experience with Apache Beam/Google Dataflow/Apache Spark in creating end to end data pipelines. Experience in some of the following: Python, Hadoop, Spark, SQL, Big Query, Big Table Cloud Storage, Datastore, Spanner, Cloud SQL, Machine Learning. Experience...

  • GCP Data Engineer

    1 month ago


    Greater Bengaluru Area, India Tredence Inc. Full time

    Have Implemented and Architected solutions on Google Cloud Platform using the components of GCPExperience with Apache Beam/Google Dataflow/Apache Spark in creating end to end data pipelines.Experience in some of the following: Python, Hadoop, Spark, SQL, Big Query, Big Table Cloud Storage, Datastore, Spanner, Cloud SQL, Machine Learning.Experience...

  • Senior Data Engineer

    3 weeks ago


    Kolkata, India FinArb Consulting Full time

    ResponsibilitiesWe are looking for a for a strong self-driven Senior Data Engineer to join the team of data and software ­­­engineering Primary responsibilities would include:   Investigate source data systems to build comprehensive data models to serve as Single Source of TruthDesign and implement flexible, robust & scalable ETL infrastructureDeploy...

  • Senior Data Engineer

    3 weeks ago


    kolkata, India FinArb Consulting Full time

    Responsibilities We are looking for a for a strong self-driven Senior Data Engineer to join the team of data and software ­­­engineering   Primary responsibilities would include:     Investigate source data systems to build comprehensive data models to serve as Single Source of Truth Design and implement flexible, robust & scalable ETL...

  • Senior Frontend Engineer

    9 minutes ago


    Greater Kolkata Area, India Energy Aspects Full time

    PurposeAre you a talented front-end engineer with a passion for building modern, user-friendly web applications? As our senior front-end engineer, you'll be responsible for spearheading the development of cutting-edge web applications that are both visually stunning and highly functional. You'll work closely with our design team to bring their vision to...

  • Data Engineer

    2 weeks ago


    Greater Bengaluru Area, India Alp Consulting Ltd. Full time

    Hello Connections Grab the Below Opportunity 👇 Job Profile: Data Engineer Experience: 3-9 Years Soft Skills – Strong Communication skills (verbal/written) Location- Bangalore NOTE :Interested Candidates kindly share Updated Resume & below details in one Email with Subject Data Engineer to get processed your application Immediately. Full Name :Email...

  • Senior Data Engineer

    3 weeks ago


    Greater Bengaluru Area, India Synechron Full time

    Job DescriptionOverall Responsibilities:Lead the development and implementation of projects using emerging technologiesMentor and guide team members to ensure the successful delivery of projectsIdentify and evaluate new technology solutions to improve business processesCollaborate with cross-functional teams to ensure alignment with the organization's...


  • Greater Noida, India Coforge Full time

    Job Description Position: Senior Data Engineer (7-10 Years) Location: Gr. Noida Duration: Full Time MUST HAVE SKILLS: 1: Python 2: Spark 3: SQL 4: Azure Cloud (Databricks) 5: AI (GPT-4, OpenAI Codex) RESPONSIBILITIES: Design, develop, and maintain data ingestion and processing pipelines for various data sources and...


  • Greater Bengaluru Area, India Alp Consulting Ltd. Full time

    Job Title: Senior Engineering Specialist Hi Applicants!!! Hiring for Job in a Reputed Organization (Leading Product Based Company) Here is a Gateway to it, through ALP Consulting. Recruiting for Senior Software Engineer Employment Type: Permanent Skills required Proficiency in technologies such as JavaBackend With Api Designing Along With Docker,Kubernetes...


  • Greater Bengaluru Area, India Mobile Premier League (MPL) Full time

    We are seeking an exceptional Data Scientist to join our team. You will be responsible for owning and executing the Data Science solutions including large dataset handling, model development and deploying the solution in real time. This is a highly impactful role where your work will help the team create the best gaming platform in the world. An ideal...


  • Greater Kolkata Area, India Estalvis Advisors Pvt. Ltd. Full time

    Hiring For A Large Retail GiantJob Title: Sr. Software EngineerJob Function: IT/Software-Web PortalExperience: 7-10 YearsCTC : Upto 14 Lacs Per AnnumQualification: Graduate with diploma in Computer application, MCA/B.Tech/BEIndustry: eCommercePrimary Skill Set: PHP, Python, JQuery, Webservice, Node JS, Microservices, REST API with understanding of SQL an...

  • Senior Data Scientist

    1 month ago


    Greater Bengaluru Area, India Novo Full time

    We are looking for a Senior Data Scientist who is enthusiastic about using data and technology to solve complex business problems. If you're passionate about leading and helping to architect and develop thoughtful data solutions, then we want to chat. Are you ready to revolutionize the small business banking industry with us?About the Role:Build and manage...

  • Senior Data Scientist

    1 month ago


    Greater Bengaluru Area, India Novo Full time

    We are looking for a Senior Data Scientist who is enthusiastic about using data and technology to solve complex business problems. If you're passionate about leading and helping to architect and develop thoughtful data solutions, then we want to chat. Are you ready to revolutionize the small business banking industry with us? About the Role: Build and...


  • Greater Bengaluru Area, India WSP in India Full time

    Senior Renewable Engineer - WindWSP iCRC is looking for an experienced and passionate Senior Renewables Engineer who will function as a senior technical consultant for utility-scale (5MW+) Wind Energy projects in Southern Africa and also be able to support the international business as required.KEY RESPONSIBILITIES:TechnicalManaging high-quality quality...

  • Senior Data Engineer

    4 weeks ago


    Kolkata, India Nilasu consulting Full time

    Summary :- We are seeking a highly skilled and experienced Senior Data Engineer to join our team.- You will play a key role in designing, developing, and maintaining large-scale data processing pipelines using Python and Spark/PySpark.- Your expertise in distributed computing frameworks and DevOps tools will be instrumental in building efficient and scalable...

  • Senior Data Analyst

    4 weeks ago


    Greater Bengaluru Area, India Nineleaps Full time

    PRIMARY RESPONSIBILITIES ● Deep dive into historical data to understand their sources, feature sets,and identify patterns. ● Write queries, perform ad hoc analysis and create reporting to ensure the team is moving towards smarter decisions. ● Performing analytical deep dives to drive product/strategy road-map. ● Ability to drive automation and...

  • Senior Data Analyst

    2 weeks ago


    Greater Bengaluru Area, India Nineleaps Full time

    PRIMARY RESPONSIBILITIES● Deep dive into historical data to understand their sources, featuresets,and identify patterns.● Write queries, perform ad hoc analysis and create reporting to ensurethe team is moving towards smarter decisions.● Performing analytical deep dives to drive product/strategy road-map.● Ability to drive automation and process...