Senior Data Engineer

6 days ago

tiruppur, India Luxoft Full time

Project Description:

As a Senior Engineer in the Data Engineering & Analytics team, you will develop data & analytics solutions that sit atop vast datasets gathered by retail stores, restaurants, banks, and other consumer-focused companies. The challenge will be to create high-performance algorithms, cutting-edge analytical techniques including machine learning and artificial intelligence, and intuitive workflows that allow our users to derive insights from big data that in turn drive their businesses. You will have the opportunity to create high-performance analytic solutions based on data sets measured in the billions of transactions and front-end visualizations to unleash the value of big data. You will have the opportunity to develop data-driven innovative analytical solutions and identify opportunities to support business and client needs in a quantitative manner and facilitate informed recommendations/decisions through activities like building ML models, automated data pipelines, designing data architecture/schema, performing jobs in big data cluster by using different execution engines and program languages such as Hive/Impala, Python, Java, Kafka, Spark, R, etc.


Responsibilities:

• Hands-on developer who writes good quality, secure code that is modular, functional, and testable.

• Drive the evolution of Data & Services products/platforms with an impact-focused on data science and engineering.

• Design and implement scalable data architecture and data pipelines

• Solving complex problems with multi-layered data sets, as well as optimizing existing machine learning libraries and frameworks.

• Provide support for deployed data applications and analytical models by being a trusted advisor to Data Scientists and other data consumers by identifying data problems and guiding issue resolution with partner Data Engineers and source data providers.

• Ensure proper data governance policies are followed by implementing or validating Data Lineage, Quality checks, classification, etc.

• Discover, ingest, and incorporate new sources of real-time, streaming, batch, and API-based data into our platform to enhance the insights we get from running tests and expand the ways and properties on which we can test Experiment with new tools to streamline the development, testing, deployment, and running of our data pipelines.

• Participate in the development of data and analytic infrastructure for product development

Continuously innovate and determine new approaches, tools, techniques & technologies to solve business problems and generate business insights & recommendations

• Partner with roles across the organization including consultants, engineering, and sales to determine the highest priority problems to solve

• Evaluate trade-offs between many possible analytics solutions to a problem, taking into account usability, technical feasibility, timelines, and differing stakeholder opinions to make a decision

Break large solutions into smaller, releasable milestones to collect data and feedback from product managers, clients, and other stakeholders

• Evangelize releases to users, incorporating feedback, and tracking usage to inform future development

Work with small, cross-functional teams to define the vision, establish team culture and processes

Consistently focus on key drivers of organization value and prioritize operational activities accordingly

• Escalate technical errors or bugs detected in project work

• Maintain awareness of relevant technical and product trends through self-learning/study, training classes, and job shadowing.

• Support the building of scaled machine learning production systems by designing pipelines and engineering infrastructure.


Mandatory Skills:

• Working proficiency in using Python/Scala, Spark (tuning jobs), SQL, Hadoop platforms to build Big Data products & platforms.

• Good programming skills in Java and spring boot and Junit.

• Knowledge in software development test approaches & frameworks

• Familiarity with RESTful APIs and micro-services architectures

• Experience in working with CI/CD

• Experience in working with SQL database like Postgres, Oracle

• Preferably with hands-on experience with Hadoop big data tools (Hive, Impala, Spark)

• Experience with data pipeline and workflow management tools: NIFI, Airflow.

• Comfortable in developing shell scripts for automation.

• Good troubleshooting and debugging skills.

• Proficient in standard software development, such as version control, testing, and deployment

• Demonstrated basic knowledge of statistical analytical techniques, coding, and data engineering

• Ability to quickly learn and implement new technologies

• Ability to Solve complex problems with multi-layered data sets

• Ability to innovate and determine new approaches & technologies to solve business problems and generate business insights & recommendations.

• Ability to multi-task and strong attention to detail

• Flexibility to work as a member of a matrix based diverse and geographically distributed project teams

• Good communication skills - both verbal and written - and strong relationship, collaboration skills, and organizational skills


Nice-to-Have Skills:

• Experience with performance Tuning of Database Schemas, Databases, SQL, ETL Jobs, and related scripts

• Experience in working with Cloud APIs (e.g., Azure, AWS)

• Experience participating in complex engineering projects in an Agile setting e.g. Scrum


Languages:

  • English: C1 Advanced

  • Sales/Business Development

    2 weeks ago

    Tiruppur, India Wissen Technology Full time

    Job Title: Sales/Business Development (Senior Manager/Director) - Data Engineering & AnalyticsLocation: BangaloreJob Type: Full-timeJob DescriptionJob Title: Sales/Business Development (Senior Manager/Director) - Data Engineering & AnalyticsJob Summary: Wissen Technology is seeking a dynamic and results-driven Sales/Business Development Director with a...

  • Data Engineer

    3 days ago

    tiruppur, India Recro Full time

    Job Title: Azure Data EngineerLocation: RemoteExperience: 4+ yearsJob Description:We are seeking a skilled Azure Data Engineer with expertise in Python, SQL, data modeling, and Azure technologies to join our team. The ideal candidate will have experience developing business data models and products, with a strong focus on data flows from ingestion to...

  • Azure Data Engineer

    3 days ago

    tiruppur, India Recro Full time

    Primary_skills(Must_have)1. SQL Expert - Build [must have]2. Data Modelling – Data structures & common algorithms. Understand E2E Data Flow from Ingestion to Exploitation [must have]3. Azure Databricks / Azure Data Lake Storage [must have]4. Azure Data Factory - orchestration of pipelines [must have]5. Azure Data Engineering Platform [must have to...

  • Freelance GCP Data Engineer

    3 weeks ago

    Tiruppur, India ThreatXIntel Full time

    Company DescriptionThreatXIntel is a startup cyber security company specializing in protecting businesses and organizations from cyber threats. Our team provides services such as cloud security, web and mobile security testing, and DevSecOps. We focus on delivering customized, affordable solutions to meet the specific needs of our clients, regardless of...

  • Freelance Backend Data Engineer

    3 days ago

    tiruppur, India Leading MNC Full time

    Looking for a Freelance Backend Data Engineer to join a team of rockstar developers. The candidate should have a minimum of 4 years. of experience .There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You will...

  • Senior QA Engineer

    4 weeks ago

    Tiruppur, India Art Technology and Software Full time

    Position OverviewThe Senior QA Engineer is responsible for executing different testing activities from preparation of test cases using user stories, use cases or requirement to testing execution until the completion of deliverables to customer as part of a scrum or development team. The Senior QA Engineer should be able to work independently for small to...

  • Senior Implementation Engineer

    4 weeks ago

    Tiruppur, India Classic Search Private Limited Full time

    Senior Engineer Implementation Industry - Semi conductor Location - Bangalore Required Years of exp - 6-14 years Qualification:- B.Tech or M.Tech in Electrical/Electronic EngineeringSkill sets Requirements: - Expertise in Static Timing Analysis, Formal Verification and Synthesis preferably with Synopsys tools and in full chip Floor planningExperience in PnR...

  • Senior Software Engineer

    1 month ago

    Tiruppur, India Arthan Full time

    We are seeking an innovative and forward-thinking Senior Software Engineer (SSE) - Knowledge Graphs, who will lead our partner organization's technical team for ontologies in shaping the technological landscape specifically around knowledge graph creation frameworks. The ideal candidate will be passionate about leveraging data-driven technologies, including...

  • Senior Engineering Manager

    2 weeks ago

    tiruppur, India Netradyne Full time

    We @Netradyne harness the power of Computer Vision and Edge Computing to revolutionize the modern-day transportation ecosystem. Netradyne is an industry leader in fleet safety solutions , immediately improving driver behavior and fleet performance, and setting commercial vehicle driving standards. Netradyne collects and analyses more data points and...

  • Data Science, Lead/ Staff Engineer

    2 weeks ago

    tiruppur, India Tejas Networks Full time

    About Tejas Networks: Tejas Networks is an R&D-driven organization focused on being at the forefront of technological advancements in the field of telecommunications. With an R&D team comprising of more than 50% of the organization strength, we have continued to invest in the development of new product capabilities. Our products include carrier-grade...

  • Data Scientist

    2 weeks ago

    Tiruppur, India Recrud Full time

    About Us:We are a fast-growing SaaS startup focused on leveraging cutting-edge AI/ML technologies to build a revolutionary platform. Our product aims to solve [specific problem] by using state-of-the-art Natural Language Processing (NLP) and Large Language Learning Models (LLLM). We are in search of a Founding Data Scientist to join our leadership team, take...

  • AI/ML Engineer

    3 days ago

    tiruppur, India Lingaro Full time

    AI/ML Engineer – Senior ConsultantAI Engineering Group is part of Data Science & AI Competency Center and is focusing technical and engineering aspects of DS/ML/AI solutions. We are looking for experienced AI/ML Engineers to join our team to help us bring AI/ML solutions into production, automate processes, and define reusable best practices and...

  • Senior Frontend Engineer

    1 month ago

    Tiruppur, India Allica Bank Full time

    About Allica BankAllica is the UK’s fastest growing company - and the fastest-growing financial technology (fintech) firm ever. Our purpose is to help established SMEs, one of the last major underserved opportunities in fintech. Established SMEs are the backbone of local communities - representing over a third of our economy - yet have been largely...

  • Data Analyst

    3 weeks ago

    Tiruppur, India techjays Full time

    Why you’ll be excited to work at Techjays:We are on a bold mission to create the best software services offering in the world to work on startups in nascent industries and greenfield projects to large-scale enterprises. As a growth-stage company, we combine a depth of capabilities and resources of our leadership with the ambition, culture, and agility of a...

  • Senior Cloud Engineer

    1 month ago

    Tiruppur, India CBTS Full time

    Senior Cloud DeveloperSeeking experienced developer & DevOps practioner, with strong cloud/systems background, to work on exciting projects building and enhancing the framework for security-as-code in a large multi-cloud environment, leveraging modern DevOps tools & processes. This is a senior role requiring hands-on development across multi-cloud...

  • Senior Software Engineer

    2 months ago

    Tiruppur, India 6thStreet.com Full time

    6THSTREET.COM is one of the largest omnichannel fashion and lifestyle destinations in the GCC, featuring over 1,200 international brands. Our platform offers collections from renowned names such as Dune London, ALDO, Naturalizer, Nine West, Charles & Keith, New Balance, Crocs, Birkenstock, Skechers, Levi’s, Aeropostale, Garage, Nike, Adidas Originals,...

  • Data Analyst

    4 weeks ago

    Tiruppur, India Mettler Toledo India Private Limited Full time

    Job Location - Powai, MumbaiDescription about Data Science Team @ IMSGUnlocking Business Insights through Data ExpertiseOur team is passionate about empowering our Strategic Business Units (SBUs) with data-driven decision making. We achieve this by:• In-depth Analysis: We analyze global sales and service data to extract valuable insights and trends.•...

  • Senior Full Stack Engineer

    4 weeks ago

    Tiruppur, India RSA Tech Full time

    Position: Senior Full Stack Engineer - React, Java, AWSWork location: Hybrid (Pune, MH (or) Chandigarh, PB)Job-Type: Full-time Experience Required: 4 yrs. Mandatory Skills: React, Typescript, JavaGood to Have Skills: Python, Sage maker, AWS Required Skills and Qualifications:   ·      Bachelor’s Degree in Computer Science or 4 years of related...

  • Senior Qlik Sense Developer

    4 weeks ago

    Tiruppur, India Technocrats Domain Inc Full time

    We need an expert in Qlik Sense who writes SQL queries well.Role DescriptionThe Senior Qlik Sense Developer will be responsible for designing, developing, implementing, and maintaining Qlik Sense dashboards and visualizations. Need strong SQL skills.QualificationsProficiency in Qlik Sense design, development, and implementationExperience in creating Qlik...

  • Senior Cloud Engineer

    2 weeks ago

    Tiruppur, India Searce Inc Full time

    Join Searce, a Modern Tech Firm, as a Sr. Cloud Engineer! (Mumbai Only)Skill - AWS or GCP, Kubernetes , DevOps & OSS. Handling multiple projectsKindly note that this opportunity is available exclusively for Mumbai locations. It's important to mention that this is an on-site position, and it involves working from the office for all 5 working days.Notice...