Senior Data Engineer

3 weeks ago

Bangalore, Karnataka, India Infimetrics Info Solutions Full time

Position : Senior Java (Data Engineer).


Reports to : Technical Lead.

Responsible for : Contributing to software development as per business requirements.

Purpose of Role :

- We are a data driven asset management firm with a very wide variety of datasets with varying frequency of updates that essentially drive our research models which ultimately determine the daily trading activity.

- We also trade a wide range of asset classes like Equities, Futures, Options, FX, Swaps, etc.

- New business requirements and enhancements to systems form a key part of our ongoing success.

- We constantly add asset classes to the trading systems, supply new data sets to our researchers, add new funds, create new risk measures, produce new reporting for portfolio managers, client services, fund accounting and improve our trade generation, execution, and back-office processes.

- This role is for a Senior Data Engineer to join the Data Development team.

- The team has a wide remit and are responsible for software changes across a range of systems including data delivery to the business and is spread across India and London.

- The role reports to the Technical Lead based in London and overlaps with UK business ours.

- Most of our existing stack is Java and MS SQL Server based.

- But we are in the process of migrating the data stack to the Cloud where Python is used more.

- We expect Java and Python to co-exist in our Technology landscape.

- The systems are proprietary and built on bespoke internal frameworks, but the direction of travel is more Open Source.

- Snowflake on AWS will be our primary data store going forward.

- We use Excel and Power BI for reporting and visualization.

- We also utilize tools like Jupyter notebooks and Ansible for release and deployment automation.

- We use JIRA for change management and follow an Agile approach to delivery.

- Your role will be to contribute to the development effort across various systems to deliver projects as per specification provided by the business and set by senior technical members of the team.

- You will also be required to provide 3rd line support to the Operations teams and provide necessary software fixes.

- The team generally tends to overlap UK hours but there would be occasional flexibility depending on need.

Primary Duties :

- Build a broad knowledge across various systems and processes via knowledge transfer and capture any missing information on the wiki.

- Gather software requirements by talking to various stakeholders and document them.

- Work closely with onshore teams and Technical Lead to ensure the technical deliveries are in line with business requirements.

- Provide necessary effort and time estimates for tasks to aid scheduling and prioritization.

- Provide necessary 3rd line support to the Operations teams including software fixes as needed.

- Work on any BAU development activity including on-boarding new data sets and delivering new markets into the system.

- Work with external vendors from time to time to gather necessary requirements for new data or changes to existing data specifications.

- Maintain pro-active communication with relevant stakeholders and keep them updated on progress.

General.

- Any other duties commensurate with the post holder's position and seniority; and.

- All employees should understand that it is their personal responsibility to comply with all organisational, statutory and regulatory policies and :


- Strong proficiency in programming languages such as Python, Scala, or Java, with a focus on data processing frameworks like Apache Spark or Apache Flink.


- Expertise in SQL and relational databases (e.g., PostgreSQL, MySQL) as well as NoSQL databases (e.g., MongoDB, Cassandra).


- Experience with distributed computing platforms and technologies such as Hadoop, HDFS, and MapReduce.


- Familiarity with cloud platforms such as AWS, Azure, or Google Cloud, and their data services (e.g., S3, Redshift, BigQuery).


- Solid understanding of data warehousing concepts and technologies (e.g., Snowflake, Amazon Redshift).


- Excellent problem-solving skills and attention to detail, with the ability to troubleshoot complex data issues.


- Strong communication and collaboration skills, with the ability to work effectively in a cross-functional team environment.


Preferred Skills :


- Experience with streaming data processing frameworks such as Apache Kafka or Apache Beam.


- Knowledge of containerization and orchestration tools such as Docker and Kubernetes.


- Understanding of machine learning concepts and frameworks (e.g., TensorFlow, PyTorch).


- Certifications in relevant data engineering technologies (e.g., AWS Certified Big Data - Specialty, Google Professional Data Engineer).

(ref:hirist.tech)

  • Senior Data Engineer

    2 months ago

    Bangalore, Karnataka, India Squareroot Consulting Pvt Ltd Full time

    We are seeking a highly skilled and experienced Senior Data Engineer to join our team.Location : Bangalore (Work from Office all 5 days)Experience : 6+ YearsRole : Senior Data EngineerThe ideal candidate will have a strong background in big data technologies and extensive hands-on experience with Cloudera, Spark, Hive, HDFS, Flink, Data Lake, and...

  • Senior Data Engineer

    3 weeks ago

    Bangalore, Karnataka, India Squareroot Consulting Pvt Ltd Full time

    We are seeking a highly skilled and experienced Senior Data Engineer to join our team.Location : Bangalore (Work from Office all 5 days)Experience : 6+ YearsRole : Senior Data EngineerThe ideal candidate will have a strong background in big data technologies and extensive hands-on experience with Cloudera, Spark, Hive, HDFS, Flink, Data Lake, and...

  • Senior Engineer

    2 months ago

    Bangalore, Karnataka, India Prudential Manpower Pvt.lTD Full time

    Immediately hiring for our client for the role of Senior Engineer Data & Analytics. Total Exp : 8+ years. Location: Anywhere in India. What you'll do:As the senior manager of the data engineering team, you will be entrusted with the delivery of scalable services that empower our customers to maximize return on brand outreach and recruitment.- Data...

  • Senior Engineer

    3 weeks ago

    Bangalore, Karnataka, India Prudential Manpower Pvt.lTD Full time

    Immediately hiring for our client for the role of Senior Engineer Data & Analytics. Total Exp : 8+ years. Location: Anywhere in India. What you'll do:As the senior manager of the data engineering team, you will be entrusted with the delivery of scalable services that empower our customers to maximize return on brand outreach and recruitment.- Data...

  • Data Engineer

    3 weeks ago

    Bangalore, Karnataka, India Magna Hire Full time

    Responsibilities :Support the data engineering team in implementing analytics solutions, ensuring adherence to project timelines and quality standards.Participate in at least one full implementation lifecycle of dbt (data build tool), contributing to design, development, and deployment phases.Assist senior team members in optimizing dbt models and...

  • Data Engineer

    2 months ago

    Bangalore, Karnataka, India Magna Hire Full time

    Responsibilities :Support the data engineering team in implementing analytics solutions, ensuring adherence to project timelines and quality standards.Participate in at least one full implementation lifecycle of dbt (data build tool), contributing to design, development, and deployment phases.Assist senior team members in optimizing dbt models and...

  • Senior Data Engineer

    2 months ago

    Bangalore, Karnataka, India TecHHire Global Full time

    Job Description :We are looking for a Senior Data engineer-GCP who can design and build production data pipelines from ingestion to consumption within a big data architecture using Python, Scala, PySpark, Java implementing data engineering, ingestion, and curation functions on GCP :Must be able to write quality code and build secure, highly available...

  • Senior Data Engineer

    3 weeks ago

    Bangalore, Karnataka, India TecHHire Global Full time

    Job Description :We are looking for a Senior Data engineer-GCP who can design and build production data pipelines from ingestion to consumption within a big data architecture using Python, Scala, PySpark, Java implementing data engineering, ingestion, and curation functions on GCP :Must be able to write quality code and build secure, highly available...

  • Senior Director

    2 months ago

    Bangalore, Karnataka, India Recruise India Consulting Full time

    Role : Senior Director - Data Engineering & AnalyticsResponsibilities :- Provide strategic leadership and vision for the data engineering and analytics function, aligning it with the overall business goals.- Lead and manage a diverse team of data engineers, data architects, data scientists, and cloud operations specialists.- Foster a culture of innovation,...

  • Senior Director

    3 weeks ago

    Bangalore, Karnataka, India Recruise India Consulting Full time

    Role : Senior Director - Data Engineering & AnalyticsResponsibilities :- Provide strategic leadership and vision for the data engineering and analytics function, aligning it with the overall business goals.- Lead and manage a diverse team of data engineers, data architects, data scientists, and cloud operations specialists.- Foster a culture of innovation,...

  • Senior Data Engineer

    1 month ago

    Bangalore, Karnataka, India Scaling Theory Full time

    As a Senior Data Engineer, you will partner with Software Engineers, Data Scientists, and Business Intelligence Engineers. You will gain a deep understanding of our services and the data they produce and become our resident expert in how to transform that data into a format that is useful for analytics and business intelligence. You will proactively help to...

  • Senior Data Engineer

    3 weeks ago

    Bangalore, Karnataka, India Scaling Theory Full time

    As a Senior Data Engineer, you will partner with Software Engineers, Data Scientists, and Business Intelligence Engineers. You will gain a deep understanding of our services and the data they produce and become our resident expert in how to transform that data into a format that is useful for analytics and business intelligence. You will proactively help to...

  • Senior Data Engineer

    3 weeks ago

    Bangalore, Karnataka, India SPN Globe Full time

    Must Have Skills : Spark streaming, Kafka, AWS, Python, Data Engineering, PySpark, CI/CD, Kubernetes, Docker, Dynamo DB.Job Description :1) Proven experience as a Senior Data Engineer.2) A well-rounded engineer with good appetite for learning and ramping up on cutting edge technologies.3) Good analytical and debugging skills with self-starting capabilities...

  • Senior Data Engineer

    2 months ago

    Bangalore, Karnataka, India SPN Globe Full time

    Must Have Skills : Spark streaming, Kafka, AWS, Python, Data Engineering, PySpark, CI/CD, Kubernetes, Docker, Dynamo DB.Job Description :1) Proven experience as a Senior Data Engineer.2) A well-rounded engineer with good appetite for learning and ramping up on cutting edge technologies.3) Good analytical and debugging skills with self-starting capabilities...

  • Senior Data Engineer

    5 days ago

    Bangalore, Karnataka, India Arting Digital Full time

    Job Description :Posting title : Senior Data Engineer Experience : 8+ Years Location : Bangalore Work mode : Hybrid Primary skills : Azure services such as Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Storage, etc.Qualification : Any Engineering/ Computers degreeRoles & Responsibilities :- Design, implement, and manage Azure-based...

  • Senior Lead

    3 weeks ago

    Bangalore, Karnataka, India Peepal Technology and Management Consultant Full time

    Key Responsibilities :Data Engineer Senior Lead role is to coordinate, and execute all activities related to the requirements interpretation, design, and implementation of Data Analytics applications. This individual will apply proven industry and technology experience as well as communication skills, problem-solving skills, and knowledge of best practices...

  • Senior Lead

    3 weeks ago

    Bangalore, Karnataka, India Peepal Technology and Management Consultant Full time

    Key Responsibilities :Data Engineer Senior Lead role is to coordinate, and execute all activities related to the requirements interpretation, design, and implementation of Data Analytics applications. This individual will apply proven industry and technology experience as well as communication skills, problem-solving skills, and knowledge of best practices...

  • Senior Data Engineer

    2 months ago

    Bangalore/Hyderabad, Karnataka, India Sam manpower services & Career LLP Full time

    As a Senior Data Engineer, you will play a critical role in our organization by leading the design, development, and maintenance of our data pipeline infrastructure. You will work closely with cross-functional teams to ensure seamless data integration and accessibility for analytical and operational purposes.Key responsibilities:- Design, construct, and...

  • Senior Data Engineer

    3 weeks ago

    Bangalore/Hyderabad, Karnataka, India Sam manpower services & Career LLP Full time

    As a Senior Data Engineer, you will play a critical role in our organization by leading the design, development, and maintenance of our data pipeline infrastructure. You will work closely with cross-functional teams to ensure seamless data integration and accessibility for analytical and operational purposes.Key responsibilities:- Design, construct, and...

  • Senior Data Engineer

    1 day ago

    Bangalore, Karnataka, India Zyoin Full time

    Mode : WFO Role Purpose :As a Senior Data Engineer, you will play a crucial role in our data infrastructure, ensuring the availability and reliability of data for our organization. You will work closely with cross functional teams to design, develop, and maintain data pipelines, optimize data storage and retrieval processes, and contribute to the overall...