[15h Left] Python Data Engineer

4 days ago


Bangalore Karnataka, India Crazy Solutions Full time

Python Data Engineer Job Title Data EngineerJob Summary We are looking for a proficient Data Engineer with expertise in Amazon Redshift Python ApacheAirflow dbt Data Build Tool API integration and AWS This role will be responsible for developingand maintaining scalable data pipelines integrating data from multiple sources and ensuring thatour data architecture supports business intelligence reporting and analytics requirements You willcollaborate with cross-functional teams to build and optimize our data infrastructure and provideclean high-quality data to the business Key Responsibilities Data Pipeline Development Build and maintain robust ETL ELT pipelines using Python ApacheAirflow and dbt to extract transform and load data from various sources into Amazon Redshift Amazon Redshift Management Design optimize and maintain Amazon Redshift clusters ensuring the warehouse is capable of handling large-scale data efficiently API Integration Develop solutions to integrate external APIs for data ingestion ensuring properdata extraction transformation and integration into our data infrastructure Data Modeling Create and maintain scalable data models in Redshift that support analytics andreporting needs including designing star and snowflake schemas for optimized querying AWS Infrastructure Management Leverage AWS services such as S3 Lambda EC2 andCloudWatch to build and maintain a scalable and cost-efficient data architecture dbt Data Build Tool Use dbt to manage and automate SQL transformations ensuring modular reusable and well-documented data transformation logic Workflow Orchestration Utilize Apache Airflow to orchestrate and automate data workflows ensuring reliable data pipelines and scheduled jobs Data Quality Testing Implement and maintain data validation checks and testing frameworksto ensure data integrity accuracy and compliance across all data pipelines Collaboration Work closely with data scientists analysts and product teams to understand dataneeds and provide technical solutions that meet business objectives Performance Optimization Tune SQL queries and manage the performance of Redshift clustersto ensure fast efficient data access and analysis Data Governance Enforce data governance policies to ensure compliance with security privacy and data quality standards throughout the data lifecycle Key Skills Qualifications Bachelors Masters degree in Computer Science Engineering Data Science or a related field 3 years of experience in data engineering with expertise in Amazon Redshift Python and AWS Strong experience with Apache Airflow for workflow scheduling and orchestration Hands-on experience with dbt Data Build Tool for managing SQL transformations and datamodels Proficiency in API development and integration including the use of RESTful APIs for dataingestion Extensive experience with AWS services such as S3 Lambda EC2 RDS and CloudWatch Expertise in data modeling concepts and designing efficient data structures e g star schemas snowflake schemas in a data warehouse environment Advanced knowledge of SQL for querying and optimizing large datasets in Redshift Experience building ETL ELT pipelines and integrating data from multiple sources includingstructured and unstructured data Familiarity with version control systems like Git and best practices for code management anddeployment automation Knowledge of data governance principles including data security privacy and quality control Preferred Qualifications Experience with real-time data processing tools such as Kafka or Kinesis Familiarity with data visualization tools like Tableau Looker or Power BI Knowledge of other data warehousing solutions like Snowflake or Google BigQuery Experience with DevOps practices for managing infrastructure and CI CD pipelines Docker Kubernetes Understanding of machine learning pipelines and how data engineering supports AI ML initiatives Soft Skills Strong analytical and problem-solving skills Ability to work independently and as part of a cross-functional team Strong written and verbal communication skills with the ability to explain technical concepts tonon-technical stakeholders Detail-oriented proactive and self-motivated with a focus on continuous improvement Strong organizational and project management skills to handle multiple tasks Job Type Payroll Categories Data Engineer Software and Web Development Cloud Architects Software and Web Development DevOps Engineers Software and Web Development Database Administrator Software and Web Development Software Engineer Software and Web Development Must have Skills Python - 3 Years Amazon Redshift - 3 Years AWS - 3 Years Apache Airflow - 1 Years ETL Extract Transform Load - 1 Years Kubernetes - 1 Years


  • Cloud Sr. Engineer

    3 weeks ago


    Bangalore, Karnataka, India NTT DATA Full time

    Req ID 275654NTT DATA Services strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a Cloud Sr Engineer to join our team in Bangalore Karn xc4 x81taka IN-KA India IN Job Title Digital...

  • 15h Left: 8 Yrs Exp

    6 days ago


    Bangalore, Karnataka, India Neorealm Full time

    Role Overview We are seeking a hands-on Python AWS Data Engineer to design develop and maintain robust ETL pipelines and cloud-based data solutions The ideal candidate will have strong expertise in Python Spark SQL and AWS services with experience in building scalable data pipelines implementing data transformations and ensuring high-quality maintainable...


  • Bangalore, India NexionPro Services Full time

    Job Title: Data Engineer Experience: 8 to 10 Years Location: Bangalore Notice Period: Immediate to 15 Days Company: Altimetrik Job Description: We are looking for an experienced Data Engineer with 8–10 years of proven expertise in data engineering and analytics. The ideal candidate will have a strong foundation in SQL, ETL processes, and large-scale...


  • Bangalore, Karnataka, India Visa Full time

    Company Description Visa is a world leader in payments and technology with over 259 billion payments transactions flowing safely between consumers merchants financial institutions and government entities in more than 200 countries and territories each year Our mission is to connect the world through the most innovative convenient reliable and secure...


  • Bangalore, Karnataka, India CrowdStrike Full time

    As a global leader in cybersecurity CrowdStrike protects the people processes and technologies that drive modern organizations Since 2011 our mission hasn t changed we re here to stop breaches and we ve redefined modern security with the world s most advanced AI-native platform We work on large scale distributed systems processing almost 3 trillion...


  • Bangalore, India Jobted IN C2 Full time

    Job Description - Grade Specific Python Developer (with CI/CD; Kubernetes) Location :-Bangalore Experience :- Minimum 5+ years Join our dynamic team as a Python Developer specializing in CI/CD and Kubernetes. This role involves designing and implementing scalable, secure, and efficient solutions using modern DevOps practices and container orchestration...


  • Bangalore, Karnataka, India ScaleneWorks People Solutions Full time

    This role contributes to the construction of the development data fabric and data strategy This role will interact with architects engineers data modelers product owners as well as other team members in Clinical Solutions and R D This role will actively participate in creating technical solutions designs implementations participate in the relentless...


  • Bangalore, Karnataka, India JPMorgan Chase Full time

    Job Category Software Engineering We have an exciting and rewarding opportunity for you to take your software engineering career to the next level As a Software Engineer III at JPMorgan Chase within the Commercial Investment Bank Payments Technology team you will be an experienced member of an agile team tasked with designing and delivering...

  • Python Developer

    3 weeks ago


    Bangalore, Karnataka, India NTT DATA Full time

    NTT DATA Services strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a Python Developer to join our team in Bangalore Karn xc4 x81taka IN-KA India IN We are looking for an experienced Python...


  • Bangalore, Karnataka, India Amazon Full time

    Description Would you like to work on one of the world s largest transactional distributed systems How about working with customers and peers from the entire range of Amazon s business on cool new features Whether you re passionate about building highly scalable and reliable systems or a software developer who likes to solve business problems Selling...