15h Left) Data Engineer

3 weeks ago

Chennai Bengaluru, India Saasguru Full time
Job Description

Roles and Responsibilities :

- Design, build, and optimize scalable ETL pipelines using Apache Airflow or similar frameworks to process and transform large datasets efficiently.
- Utilize Spark (PySpark), Kafka, Flink, or similar tools to enable distributed data processing and real-time streaming solutions.
- Deploy, manage, and optimize data infrastructure on cloud platforms such as AWS, GCP, or Azure, ensuring security, scalability, and cost-effectiveness.
- Design and implement robust data models, ensuring data consistency, integrity, and performance across warehouses and lakes.
- Enhance query performance through indexing, partitioning, and tuning techniques for large-scale datasets.
- Manage cloud-based storage solutions (Amazon S3, Google Cloud Storage, Azure Blob Storage) and ensure data governance, security, and compliance.
- Work closely with data scientists, analysts, and software engineers to support data-driven decision-making, while maintaining thorough documentation of data processes.

Ideal candidate

- Strong proficiency in Python and SQL, with additional experience in languages such as Java or Scala.
- Hands-on experience with frameworks like Spark (PySpark), Kafka, Apache Hudi, Iceberg, Apache Flink, or similar tools for distributed data processing and real-time streaming.
- Familiarity with cloud platforms like AWS, Google Cloud Platform (GCP), or Microsoft Azure for building and managing data infrastructure.
- Strong understanding of data warehousing concepts and data modeling principles.
- Experience with ETL tools such as Apache Airflow or comparable data transformation frameworks.
- Proficiency in working with data lakes and cloud based storage solutions like Amazon S3, Google Cloud Storage, or Azure Blob Storage.
- Expertise in Git for version control and collaborative coding.
- Expertise in performance tuning for large-scale data processing, including partitioning, indexing, and query optimization.

  • 15 Hour Left: Experienced Billing Specialist

    2 weeks ago

    Bengaluru, Karnataka, India beBeeBilling Full time ₹ 9,00,000 - ₹ 12,00,000

    Job Title15h Left: Billing Specialist### Job OverviewAs a skilled billing specialist, you will be responsible for generating, validating, and processing invoices for real estate transactions. You will ensure accuracy and compliance with contractual agreements, payment schedules, and company policies.### Key Responsibilities- Generate, validate, and process...

  • (15h Left) Data Engineer

    4 days ago

    Bengaluru, India NTT Data Full time

    Job Description Req ID:334009 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer (Talend &Pyspark) to join our team in Bangalore, Karntaka (IN-KA), India (IN). Job...

  • 15h Left: Data Engineer

    5 days ago

    Bengaluru, Karnataka, India Embitel Technologies Full time

    Job DescriptionJob Description:Data Engineer - Automotive Analytics(PySpark, Databricks, Azure, Power BI)- Data Processing Engineering - Build and optimizeETL pipelinesusingPySpark Databricksfor large-scale automotive datasets and should have implementation knowledge ofCI/CD pipelines.- Cloud Big Data - Work withAzure Data Lake, Synapse, ADF, and...

  • 15h Left: Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India HCLTech Full time

    Job DescriptionHiring for Data QA Engineer in HCLTechPrimary skill- Python, Pyspark, Postgres, Data Pipeline QALocation- Bangalore, Noida, Chennai, Hyderabad, PuneNotice Period- Only Immediate joinersExperience- 5 to 7 YearsInterested candidates share your resume to [Confidential Information]Develop and execute test strategies focused on data pipelines,...

  • 15h Left: Google Data Engineering

    2 weeks ago

    Chennai, Tamil Nadu, India Diverse Lynx Full time

    Hiring for Google Data Engineering - Chennai - We are looking for a skilled and motivated Data Engineer with 4 years of experience including strong hands-on expertise in the Google Cloud Platform GCP As a Data Engineer you will be responsible for designing developing and managing scalable data pipelines transforming raw data into actionable...

  • 15h Left Data Engineer

    1 week ago

    Bengaluru, Karnataka, India Skillmine Technology Full time

    Job DescriptionRequired experience, skills6+ years of hands-on data engineering development experience in PythonAdvanced knowledge of Python and Linux shell scriptingProficiency in SQL and Data Structures, experience in handling (transformations, read / write) huge volume of data using PythonExperience in performance improvement on data pipelines and a good...

  • 15h Left: Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Docusign Full time

    Job DescriptionCompany OverviewDocusign brings agreements to life. Over 1.5 million customers and more than a billion people in over 180 countries use Docusign solutions to accelerate the process of doing business and simplify peoples lives. With intelligent agreement management, Docusign unleashes business-critical data that is trapped inside of documents....

  • 15h Left Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India AMISEQ Full time

    Job Role:- Data EngineerLocation: BangaloreExperience: 3- 6 yearsRequired skillsKafka, Flink, JavaBig Data experience- 3-4 years'3 years of hands-on experience supporting Java-based applicationsExperience working with Apache Flink, Apache Spark, or other distributed data processing frameworksUnderstanding of production concerns like data consistency,...

  • 15h Left Senior Network Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time

    Job DescriptionMake an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion its a place where you can grow, belong and thrive.Your day at NTT DATAThe...

  • 15h Left Data

    1 week ago

    Bengaluru, Karnataka, India MUFG Full time

    About MUFG Global Service MGS MUFG Bank Ltd is Japan s premier bank with a global network spanning in more than 40 markets Outside of Japan the bank offers an extensive scope of commercial and investment banking products and services to businesses governments and individuals worldwide MUFG Bank s parent Mitsubishi UFJ Financial Group Inc MUFG ...