Data Engineer III

20 hours ago

Chennai, Tamil Nadu, India PDI Technologies Full time ₹ 12,00,000 - ₹ 36,00,000 per year
For nearly 40 years, PDI has helped convenience retailers and petroleum wholesalers adapt to changes in the industry by leveraging the latest technologies. Simplifying the complexity in your world is our main focus. That's why we're delivering an integrated portfolio of global, cloud-based solutions and services to meet our customers' needs today and well into the future. From the back office to fuel logistics, and the latest wave of consumer demands and digital commerce, PDI solutions and services are designed to give you a competitive edge. Our deep domain knowledge is backed by our industry-specific software offerings and a people-first approach. Serving the entire supply chain means that we've had to solve a wide range of industry challenges and deliver measurable value. To address the increasingly dynamic convenience retail and petroleum wholesale industry, PDI leads the way with converged digital, cloud-optimized, secure, and scalable solutions including ERP, Loyalty, Fuel Pricing, Point-of-Sale and Logistics to create a broader industry ecosystem. The result is unparalleled insights, simplified operations, and higher profitability for your employees and business. We're dedicated to helping our more than 1,500 customers in over 50 countries, providing solutions and services to over 200,000 sites.
The Opportunity: If you love to develop great quality software and are eager to learn new technologies and develop new skills, then we have a great opportunity for you: join our PDI family and work closely with other talented PDI engineers to deliver solutions that delight our customers every day
As a Data Engineer III, you will be part of an agile data services team responsible for developing and maintaining our industry-leading cloud-based big data and data analytics infrastructure serving major global fortune 500 companies. The candidate will help with design, development, unit testing, performance testing, deployment, troubleshooting, defect resolution, and support of our data services infrastructure.  Key Responsibilities:
  • Utilize architectural and design skills for a variety of data-oriented tasks.
  • Establish and maintain strong relationships with business stakeholders.
  • Build highly reliable & scalable data systems with good data governance that provide complete, secure, accurate and timely data for analytics.
  • Expertise in data & dimensional modeling with good business acumen in supporting new product features and services - including financial reporting.
  • Assist development and analytics organization to meet its delivery goals with regards to data design, flow and infrastructure.
  • Perform Data Modeling, Big data architecture design, support and implementation of OLTP, real time OLAP/data warehouse systems and ETL pipelines.
  • Track/Build & monitor data/system KPI's like storage, performance & usage
  • Hands on administration of big data databases/compute engines (Redshift, EMR, Spark or similar) and optimization/tuning techniques
  • Prepare required documentation as outlined by departmental standards.
  • Identify, analyze and back track data problems as needed.
  • Provide recommendations for application and system improvements.
  • Mentor other team members
  • Work with business users to understand business requirements, issues and business and/or client processes.
  • Develop, test, and maintain high-performance of our data systems to meet the requirements of the business and/clients while adhering to departmental standards.
  • Perform quality assurance testing for all work performed.
  • Meet with agile teams as required to define and document application requirements.
  • Follow project development, deployment process & maintain industry standards and best coding practices.
  • Maintain security and organization of the company's data.
  • Provide off-hour support as assigned.
  • Gather business requirements/features, estimate the tasks, design & develop scalable solutions.
  • Work with manager to ascertain the company's data requirements.
  • Plan work to meet project deadlines, accommodate demands by development teams, set priorities and escalate issues appropriately.
  • Provide recommendations to development teams for data restructuring and performing complex maintenance.
  • Deploy and manage changes in development, staging and production.
  • Assist development and dev ops teams on SQL queries and tuning.
Qualifications:
  • At least 7 years' experience in data engineering, Data modeling, SQL, ETL, Big database, Data warehousing & basic-level administration of data infrastructure
  • At least 7 years coding experience in Java, Python, R or other equivalent programming language
  • At least 4 years big data experience
  • At least 2 years' experience as technical lead architecting data lake/data warehouse/lake house systems & mentoring team of Data Engineers / DBAs
  • Proficient in a variety of big-data tools and technologies, including Hadoop, Spark, etc., handling different storage formats like Parquet, Delta/Apache Hudi/Apache Iceberg, & managing/working with different OLTP systems & data ingestion mechanisms.
  • Experience building large scale data lakes & warehouse systems leveraging cloud environments including AWS technologies –S3, Redshift, Aurora/RDS, DMS, EC2, Glue spark, Cloud watch, & EMR
  • Experience with data orchestration tools like Airflow
  • Experience in building pipelines, ingesting data from heterogeneous systems onto data lake environments in real time
  • Proficient in Python, SQL and PL-SQL, query tuning, optimization, ETL, ELT, and Data Warehousing
  • Proficient in Business Intelligence (BI), analytic database and products, able to take business requirements and translate into database tables, pipeline design and tasks.
  • General data-specific concepts, frameworks and processes
  • Agile development practices
  • Working within an SDLC
  • Designing data warehouses including definition, structure, documentation, maintenance, long-range requirements, operational guidelines, and protection
  • Data governance, meta data management, data security measures, retention & archival across data lake layers
Other Skills and Experience :
  • Cloud experience, such as AWS, GCP or Azure
  • Building/Managing data lake & lake house architectures.
  • Experience working on different data lake storage formats.
  • Excellent oral and written communication
  • Multi-tasking and managing multiple priorities.
  • Working well in a team-oriented, collaborative environment with people from different disciplines and varying degrees of technical experience
  • Working in an Agile team environment.
Behavioral Competencies:
  • Ensures Accountability
  • Manages Complexity
  • Communicates Effectively
  • Balances Stakeholders
  • Collaborates Effectively
PDI is committed to offering a well-rounded benefits program, designed to support and care for you, and your family throughout your life and career.  This includes a competitive salary, market-competitive benefits, and a quarterly perks program. We encourage a good work-life balance with ample time off [time away] and, where appropriate, hybrid working arrangements.  Employees have access to continuous learning, professional certifications, and leadership development opportunities. Our global culture fosters diversity, inclusion, and values authenticity, trust, curiosity, and diversity of thought, ensuring a supportive environment for all. We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.

  • Sofware Engineer III

    1 week ago

    Chennai, Tamil Nadu, India RELX Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Software Engineer IIIJoin Our Team at Cirium – Shaping the Future of Aviation AnalyticsAre you a skilled developer with a collaborative spirit and a passion for making a positive impact?About The BusinessAt Cirium, we are committed to connecting communities across the globe. As a leader in aviation analytics, we empower our customers—ranging from...

  • Senior Software Engineer III

    1 week ago

    Chennai, Tamil Nadu, India Photon Full time ₹ 1,50,000 - ₹ 15,00,000 per year

    Sr. Software Engineer III - Java, Spring Boot, API - ChennaiExperience: 8 To 10 Years of Experience.Requirement: Sr. Software Engineer III (Java/Spring Boot/API Development)Looking for a Java Sr. Software Engineer with experience in building high-performing, scalable, enterprise-grade applications and you will be part of a talented software...

  • Data Engineer III

    6 days ago

    Chennai, Tamil Nadu, India Bank of America Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Description Data Engineer III - GBS IND Location: Chennai, India: About Us At Bank of America, we are guided by a common purpose to help make financial lives better through the power of every connection. Responsible Growth is how we run our company and how we deliver for our clients, teammates, communities, and shareholders every day. One of the keys...

  • Senior Software Engineer III

    21 hours ago

    Chennai, Tamil Nadu, India Photon Group Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Description Greetings EveryoneSr. Software Engineer III - Java, Spring Boot, API - ChennaiExperience: 8 To 10 Years of Experience.Requirement: Sr. Software Engineer III (Java/Spring Boot/API Development)Looking for a Java Sr. Software Engineer with experience in building high-performing, scalable, enterprise-grade applications and you will be part of a...

  • Sofware Engineer III

    1 week ago

    Chennai, Tamil Nadu, India LexisNexis Risk Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Software Engineer IIIJoin Our Team at Cirium – Shaping the Future of Aviation AnalyticsAre you a skilled developer with a collaborative spirit and a passion for making a positive impact?About the BusinessAt Cirium, we are committed to connecting communities across the globe. As a leader in aviation analytics, we empower our customers—ranging from...

  • Data Engineer

    3 days ago

    Chennai, Tamil Nadu, India Crayon Data Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Data Engineer – Chennai Build the foundation of data intelligence — design, develop, and optimize data pipelines that power AI-driven insights. Location: Chennai, India Experience: 2–3 years in data engineering or data-driven product development Role Overview As a Data Engineer, you will be responsible for building and...

  • Sofware Engineer III

    1 week ago

    Chennai, Tamil Nadu, India Renishaw Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Software Engineer IIIJoin Our Team at Cirium – Shaping the Future of Aviation Analytics Are you a skilled developer with a collaborative spirit and a passion for making a positive impact? About the BusinessAt Cirium, we are committed to connecting communities across the globe. As a leader in aviation analytics, we empower our customers—ranging from...

  • Software Engineer III

    6 days ago

    Chennai, Tamil Nadu, India Renishaw Full time ₹ 6,00,000 - ₹ 12,00,000 per year

    Job Title: Software Engineer IIILocation: ChennaiEmployment Type: Full-timeExperience Level: 4–6 yearsSkillset: We are seeking a highly motivated and skilled candidate with strong experience in Python, PySpark, SQL,AWS Glue and modern experience in data platforms like Vertica and SnowflakeKey Responsibilities:Develop and maintain scalable ETL/ELT data...

  • Data Engineer

    2 days ago

    Chennai, Tamil Nadu, India CX Data Labs Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    HiRole - Azure Data EngineerLocation - Chennai, Gurugram (5days Office)Full timeShift timing 2pm to 11pmKey Responsibilities:· Develop, maintain, and optimize scalable ETL/ELT pipelines using PySpark and Databricks.· Collaborate with cross-functional teams to design and implement data models and data integration solutions.· Create and maintain robust SQL...

  • Systems Engineer III

    2 weeks ago

    Chennai, Tamil Nadu, India Elsevier Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Would you like to be part of a team that delivers high-quality software to our customers?Are you a highly visible champion with a 'can do' attitude and enthusiasm that inspires others?About Our TeamOur global team supports products that educate and provide electronic health records, introducing students to digital charting and preparing them to document care...