Senior Data Engineer

2 months ago

Solapur, India Visa Full time

Job Description

The Senior Data Engineer role involves leading and mentoring a team of data engineers, coordinating with other teams to manage and prioritize projects, and driving the strategy, roadmap, and execution of key data engineering initiatives. The role requires understanding and translating business needs into data models, creating robust data pipelines, and developing and maintaining data lakes. It also involves implementing and maintaining CI/CD pipelines for data solutions. The candidate should be able to define and manage data load procedures, implement data strategies, and ensure robust operational data management systems. Collaborating with stakeholders across the organization to understand their data needs and deliver solutions is also a key part of this role. The ideal candidate will be proficient in big data tools like Hadoop, Hive, and Spark, programming languages such as Scala, Python, SQL, and have strong analytical skills related to working with structured and unstructured datasets.


Essential Functions:

  • Experience in Requirement Gathering, Estimating, Managing large scale Data Engineering Projects.
  • Requirement Analysis: Understand and translate business needs into data models supporting long-term solutions.
  • Data Modeling: Work with the Business team to implement data strategies, build data flows and develop conceptual data models.
  • Data Pipeline Design: Create robust and scalable data pipelines and data products in a variety of domains.
  • Data Integration: Develop and maintain data lakes by acquiring data from primary and secondary sources and build scripts that will make our data evaluation process more flexible or scalable across data sets.
  • Testing: Define and manage the data load procedures to reject or discard datasets or data points that do not meet the defined business rules and quality thresholds.
  • Deployment: Implement data strategies and develop physical data models, along with the development teams, data analyst teams and information system team to ensure robust operational data management systems.
  • Understanding of and ability to Implement Data Engineering principles and best practices.
  • Team Leadership: Lead and mentor a team of data engineers. Coordinate with other teams to manage and prioritize projects. Drive strategy, roadmap, and execution of key data engineering initiatives.
  • Stakeholder Management: Collaborate with stakeholders across the organization to understand their data needs and deliver solutions.
  • Continuous Integration and Continuous Deployment (CI/CD): Implement and maintain CI/CD pipelines for data solutions, ensuring rapid, reliable, and streamlined updates to the data environment.


Qualifications

Technical Skills:

• Extensive experience in big data tools: Hadoop, Hive, and Spark.

• Proficiency in Scala, Python, SQL, and PySpark.

• Experience with Unix/Linux systems with scripting experience in Bash.

• Experience with data pipeline and workflow management tools like Airflow, etc.

• Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.

• Experience with cloud services AWS or Azure.

• Experience with stream-processing systems: Kafka, Spark-Streaming, etc.

• Strong analytic skills related to working with structured & unstructured datasets.

• Proficiency in managing and communicating data warehouse plans to internal clients.

• Experience with building processes supporting data transformation, data structures, metadata, dependency, and workload management.

• A successful history of manipulating, processing, and extracting value from large, disconnected datasets.


Other Skills:

• Strong problem-solving skills.

• Excellent communication skills.

• Ability to work in a team.

• Detail-oriented and excellent organizational skills.


• 8+ years of work experience with a bachelor’s degree or at least 5+ years of work experience with an Advanced degree (e.g. Master’s, MBA, JD, MD) or 3+ years of work experience with a PhD degree

• Exposure to Financial Services/ Payments Industry

• Proven leadership skills with experience leading a team of data engineers.

Leadership Competencies

• Exhibits intellectual curiosity and a desire for continuous learning.

• Demonstrates integrity, maturity, and a constructive approach to business challenges.

• Role model for the organization and implementing core Visa Values

• Respect for the Individuals at all levels in the workplace

• Strive for Excellence and extraordinary results.

• Use sound insights and judgments to make informed decisions in line with business strategy and needs.

• Leadership skills include an ability to allocate tasks and resources across multiple lines of businesses and geographies. Leadership extends to ability to influence senior management within and outside Analytics groups.

• Ability to successfully persuade/influence internal stakeholders for building best-in-class solutions.

  • Senior Data Engineer

    2 days ago

    Solapur, India VidPro Consultancy Services Full time

    Role: Senior Databricks Engineer / Databricks Technical Lead/ Data ArchitectExperience: 3-15 yearsLocation: Bangalore, Chennai, Delhi, Pune, Kolkata and GurgaonMandatory Skills: Python/ PySpark / Spark with Azure/ Azure Databricks / Azure DataFactoryPrimary Roles and Responsibilities:● Developing Modern Data Warehouse solutions using Databricks and AWS/...

  • Senior Data Engineer

    3 weeks ago

    Solapur, India KCS iT Full time

    We’re looking for the special, unique and amazing YOU!@ KCS IT, we look for the ones that stands out, for those that always wants to be better and fight for it, and for those who has the same values that we do: dedication, energy, integrity, transparency, flexibility, trust, honesty, hard work, proactivity, team work.At KCS we stand for equality and value...

  • Data Engineer

    3 weeks ago

    Solapur, India Truelancer.com Full time

    Job Title: Sr Data EngineerExperience: 5 - 9 yrsLocation: Work from Office ,ChennaiMandatory Skills: AWS Data Services+Strong in SQL Queries As a Senior Data Engineer, you will play a crucial role in designing, developing, and maintaining data pipelines and analytics solutions. You will work with a range of AWS technologies and data tools to ensure our data...

  • Senior Backend Engineer

    1 week ago

    Solapur, India Pictory Full time

    LocationIndia (Remote)Why join:Join our tight-knit software engineering team as a Senior Software Engineer (backend), where you'll play a pivotal role in crafting exceptional products. You'll have the opportunity to lead and mentor junior engineers, while contributing your expertise to shape the development of robust and efficient API solutions.Impact:Lead...

  • Senior Software Engineer

    1 month ago

    Solapur, India HiroJet Full time

    Senior/Staff Software Engineer (Deep ClickHouse Expertise)Location - Remote About The Company We are a Bay-Area (California) based security data platform that enables teams to be more effective by building the infrastructure modern security teams need to most effectively use their security data. Through its AI-enabled “Abstract Security Engineer” (ASE),...

  • Senior Software Engineer

    7 days ago

    Solapur, India TerraFinn Technologies Full time

    Company Description TerraFinn Technologies outsources scholars from various top business schools in India and experienced employees to Fintech companies globally. The company focuses on providing expert employees to help fintech companies grow their business and achieve their goals efficiently. TerraFinn Technologies values long-term relationships with both...

  • Data Engineer

    1 month ago

    Solapur, India University of the People Full time

    Vandey provides outsourced instructional services to institutions of higher education and is seeking a candidate to work for Vandey to serve in a full-time role as a Data Engineer with the University of the People.University of the People (UoPeople) is the first non-profit, tuition-free, American, accredited, 100% online university. Uopeople is a leading...

  • Data Engineer

    4 weeks ago

    Solapur, India Bahwan CyberTek Full time

    We are hiring Data Engineers in Chennai, Bangalore, Coimbatore, Pune, and Mumbai locations.Shift: Mid-Shift (2PM - 11PM)Required Skill:Minimum 5+ years of experience as Data EngineerADF ( azure data factory) mandatory Snowflake Mandatory Snowflake admin knowledge - PreferableLooking for immediate joiners

  • Data Engineer

    2 weeks ago

    Solapur, India Response Informatics Full time

    LOCATION- Pune/BangaloreEXPERIENCE- 5+YEARSGCP Data Engineer :Experience: 5+ years of experience in data engineering, with a focus on cloud-basedsolutions. Extensive experience with Google Cloud Platform (GCP) and its data services,includingBigQuery, Dataflow, Pub/Sub, Cloud Storage, and Cloud Composer. Experience with ETL tools and processes....

  • Senior Data Scientist

    2 months ago

    Solapur, India MeraDoc Full time

    Roles & Responsibilities:As a Senior Data Scientist at MeraDoc, you will play a pivotal role in managing and leveraging our extensive data assets to drive business and product decisions. Your key responsibilities will include:1.⁠ ⁠Data Assessment and Structuring: - Assess the current state of data storage, organization, and integration across multiple...

  • Data Engineer

    7 days ago

    solapur, India Greenway Health Full time

    Minimum Qualifications · 5 years’ experience in Data Engineering – Data Processing using various tools · Excellent understanding and hands on experience with T-SQL programming · Excellent understanding and hands on experience with Microsoft SQL Server, other RDBMS, and No-SQL systems · Excellent understanding and hands on experience with ETL, Data...

  • Senior Software Engineer

    4 weeks ago

    Solapur, India RealStar AI Full time

    Senior Backend & AI EngineerRealStar AI - Building an AI Real Estate AgentAbout Us: RealStar AI is building the world's first AI Real Estate Agent, transforming how people buy and sell homes without paying traditional commissions. We're on a mission to provide a superior, seamless, and cost-effective alternative to traditional real estate services by...

  • Python Data Engineer

    2 weeks ago

    solapur, India CredHive Full time

    Position Overview We are a seed-funded startup focused on using state-of-the-art AI technologies to revolutionize the credit industry. Our team consists of experts in machine learning and software engineers who have worked at top-tier US tech companies like Apple, Amazon, etc , and we are passionate about using AI to improve access to credit information for...

  • Data Engineer

    4 weeks ago

    Solapur, India Response Informatics Full time

    JD for Data EngineerEXPERIENCE- 6-10YRS Location – Noida Design and maintain efficient data pipelines for seamless ETL operations. Architect scalable data warehouses and optimize data storage systems. Develop and deploy data models and dashboards to support business decision-making. Collaborate with cross-functional teams to understand data...

  • Senior Search Engine Optimization Specialist

    1 week ago

    Solapur, India enqos Full time

    Join our dynamic team as an Senior SEO Specialist and drive our online presence to new heights! We are looking for a detail-oriented, data-driven professional with a passion for optimizing web content and enhancing search engine rankings Key Responsibilities: - Develop and execute effective SEO strategies to improve website rankings, drive traffic, and...

  • Senior AI Engineer

    4 days ago

    solapur, India Infinite Possibilities Full time

    Job descriptionCompany DescriptionInfinite Possibilities is a Generative AI startup that offers full-range product development services for software vendors and corporations. We specialize in accelerating AI adoption and AI-driven value creation. Currently, we are focused on developing innovative applications for the FinTech sector. We are also working on...

  • Senior Locomotive Controls Engineer

    3 months ago

    Solapur, India Voltify Full time

    Mission:Voltify is the first and only company offering an end-to-end solution for freight rail decarbonization. As our Senior Locomotive Electrical Engineer, you'll play a pivotal role in shaping the future of sustainable transportation by spearheading the creation of our locomotive Voltify computerOur leadership team boasts decades of experience, having...

  • Senior Platform Engineer, AI Server

    1 month ago

    Solapur, India Rexus Group Full time

    Job Description: Senior Platform Software Engineer, AI Server - GPULocation: Mumbai, Powai (Rotational Shift)Job Type: Full-timeExperience: 15+ YearsAbout Rexus Group: Rexus Group is a leading global provider of IT solutions, specializing in advanced technology infrastructure, AI-driven innovations, and comprehensive managed services. We work with industry...

  • Senior/Lead Frontend Engineer

    7 days ago

    Solapur, India Eltropy Full time

    We are searching for a seasoned Senior/Lead Frontend Engineer who will be responsible for designing and developing user interfaces, ensuring their responsiveness, functionality, and overall user experience. The ideal candidate should have 6-9 years of experience and expertise in JavaScript, HTML, CSS, and React. Knowledge of frontend frameworks such as Ant...

  • Senior Staff Engineer

    2 months ago

    Solapur, India Mulya Technologies Full time

    Senior Staff Engineer - RTL DesignLocation: Bangalore /Pune We enable tomorrow’s future by accelerating the critical data communication at the heart of our digital world – from seamless video streaming to AI to the metaverse and much more. Our technology powers product innovation in the most data-demanding industries today, including data centers,...