21,276 Senior Data Engineer jobs in India
Senior Data Engineer
Job Viewed
Job Description
Position Overview
As a Senior Data Engineer (multiple openings), you will work independently to design strategies for enterprise database systems and set standards for operations, programming, and security. Critical at all stages of the data science process, Data Engineers work cross-functionally with both external and internal teams – from business analysts to data scientists; web app developers to platform engineers; IT teams to high-level executives.
Essential Duties & Responsibilities:
- Will Design and construct large relational databases.
- Integrate new systems with existing warehouse structure and refine system performance and functionality.
- Will serve as a data expert who understands how a client’s data fits into our Industry Intelligent data models.
- Utilizing this knowledge and the industry’s newest technologies, will create high performance databases that become the very foundation of the work we do.
- Will build and own “one source of truth” data sets to facilitate consistency and efficiency in extracting and analyzing data from disparate data sources.
- Will ensure data integrity by developing and executing necessary processes and controls around the flow of data.
- Will apply mathematical skills and knowledge in combination with computer science and programming skills and knowledge to innovate and improve efficiency of managing data to allow for greater speed and accuracy of producing analyses, metrics, and insights.
- Will collaborate with internal and external teams to understand business needs/issues, troubleshoot problems, conduct root cause analysis, and develop cost effective resolutions for data anomalies.
- Will provide input into data governance initiatives to enhance current systems, ensure development of efficient application systems, influence the development of data policy, and support overall corporate and business goals.
- In the performance of these complex and highly technical responsibilities, will use technology to analyze data from applicable systems to review data processes, identify issues, and determine actions to resolve or escalate problems that require data, system, or process improvement.
- Will verify the accuracy of table changes and data transformation processes as well as test changes prior to deployment as appropriate.
- Will be called on to recommend and implement enhancements that standardize and streamline processes, assure data quality and reliability, and reduce processing time to meet client expectations.
- Will stay up-to-date on data engineering and data science trends and developments and will follow company policy and procedures which protect sensitive data and maintain compliance with established security standards and best practices.
- Will perform other related professional responsibilities as may be needed.
Required Skills:
- Requires Bachelor’s degree/foreign equivalent in Computer Science, Computer Engineering, Mathematics, or a related field.
- Requires 5 years of work experience, to include:
- 5 years of experience with relational database structures, data lake management, and SQL;
- 5 years of experience in data architecture designing and programming ETL procedures;
- 5 years of experience working with commercial relational database systems (such as electronic medical records or other clinical systems, client relationship management software, sales and online marketing data systems, or accounting systems);
- 5 years of experience with one of the following: PHP, Java, or Python;
- 5 years of experience with data management methodologies and data quality assurance practices; and
- Ability to professionally communicate ideas (verbal and written).
- Experience may be gained concurrently.
- May also hold a Master’s degree/foreign equivalent in above areas with 3 years of above experience.
What's in it for You?
- Opportunity to work with a rapidly expanding tech company in the booming field of data analytics alongside some of the brightest minds in the industry
- Opportunity to work with cutting-edge technology in a casual, fun environment
- Opportunity to be a part of a local company committed to making a difference in our community
- Competitive salary and benefits package including health, vision, dental and life insurance and 401(k) plan.
Job No Longer Available
This position is no longer listed on WhatJobs. The employer may be reviewing applications, filled the role, or has removed the listing.
However, we have similar jobs available for you below.
Big Data Engineer
Posted 6 days ago
Job Viewed
Job Description
**Responsibilities:**
+ Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements
+ Identify and analyze issues, make recommendations, and implement solutions
+ Utilize knowledge of business processes, system processes, and industry standards to solve complex issues
+ Analyze information and make evaluative judgements to recommend solutions and improvements
+ Conduct testing and debugging, utilize script tools, and write basic code for design specifications
+ Assess applicability of similar experiences and evaluate options under circumstances not covered by procedures
+ Develop working knowledge of Citi's information systems, procedures, standards, client server application development, network operations, database administration, systems administration, data center operations, and PC-based applications
+ Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
**Qualifications:**
+ 3 to 5 years of relevant experience
+ Experience in programming/debugging used in business applications
+ Working knowledge of industry practice and standards
+ Comprehensive knowledge of specific business area for application development
+ Working knowledge of program languages
+ Consistently demonstrates clear and concise written and verbal communication
**Education:**
+ Bachelor's degree/University degree or equivalent experience
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
Additional Job Description
We are looking for a Big Data Engineer that will work on the collecting, storing, processing, and analyzing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company.
Responsibilities
- Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
- Implementing data wrangling, scarping, cleaning using both Java or Python
Strong experience on data structure.
Skills and Qualifications
- Proficient understanding of distributed computing principles
- Proficient in Java or Python and some part of machine learning
- Proficiency with Hadoop v2, MapReduce, HDFS, Pyspark, Spark
- Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
- Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
- Experience with Spark
- Experience with integration of data from multiple data sources
- Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
- Knowledge of various ETL techniques and frameworks, such as Flume
- Experience with various messaging systems, such as Kafka or RabbitMQ
- Experience with Big Data ML toolkits, such as Mahout, SparkML, or H2O
- Good understanding of Lambda Architecture, along with its advantages and drawbacks
- Experience with Cloudera/MapR/Hortonworks
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
---
**Job Family Group:**
Technology
---
**Job Family:**
Applications Development
---
**Time Type:**
Full time
---
**Most Relevant Skills**
Please see the requirements listed above.
---
**Other Relevant Skills**
For complementary skills, please see above and/or contact the recruiter.
---
_Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law._
_If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review_ _Accessibility at Citi ( _._
_View Citi's_ _EEO Policy Statement ( _and the_ _Know Your Rights ( _poster._
Citi is an equal opportunity and affirmative action employer.
Minority/Female/Veteran/Individuals with Disabilities/Sexual Orientation/Gender Identity.
Big Data Engineer
Posted 7 days ago
Job Viewed
Job Description
Big Data Engineer (PySpark)
Location: Pune/Nagpur (WFO)
Experience: 8 - 12 Years
Employment Type: Full-time
Job Overview
We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance and reliability.
Key Responsibilities
- Design, develop, and maintain data pipelines using PySpark and related Big Data technologies.
- Work with HDFS, Hive, Sqoop , and other tools in the Hadoop ecosystem.
- Write efficient HiveQL and SQL queries to handle large-scale datasets.
- Perform performance tuning and optimization of distributed data systems.
- Collaborate with cross-functional teams in an Agile environment to deliver high-quality solutions.
- Manage and schedule workflows using Apache Airflow or Oozie .
- Troubleshoot and resolve issues in data pipelines to ensure reliability and accuracy.
Required Skills
- Proven experience in Big Data Engineering with a focus on PySpark.
- Strong knowledge of HDFS, Hive, Sqoop , and related tools.
- Proficiency in SQL/HiveQL for large datasets.
- Expertise in performance tuning and optimization of distributed systems.
- Familiarity with Agile methodology and collaborative team practices.
- Experience with workflow orchestration tools (Airflow/Oozie ).
- Strong problem-solving, analytical, and communication skills.
Good to Have
- Knowledge of data modeling and data warehousing concepts.
- Exposure to DevOps practices and CI/CD pipelines for data engineering.
- Experience with other Big Data frameworks such as Spark Streaming or Kafka .
Big Data Engineer
Posted today
Job Viewed
Job Description
Big Data Engineer (PySpark)
Location: Pune/Nagpur (WFO)
Experience: 8 - 12 Years
Employment Type: Full-time
Job Overview
We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance and reliability.
Key Responsibilities
- Design, develop, and maintain data pipelines using PySpark and related Big Data technologies.
- Work with HDFS, Hive, Sqoop, and other tools in the Hadoop ecosystem.
- Write efficient HiveQL and SQL queries to handle large-scale datasets.
- Perform performance tuning and optimization of distributed data systems.
- Collaborate with cross-functional teams in an Agile environment to deliver high-quality solutions.
- Manage and schedule workflows using Apache Airflow or Oozie.
- Troubleshoot and resolve issues in data pipelines to ensure reliability and accuracy.
Required Skills
- Proven experience in Big Data Engineering with a focus on PySpark.
- Strong knowledge of HDFS, Hive, Sqoop, and related tools.
- Proficiency in SQL/HiveQL for large datasets.
- Expertise in performance tuning and optimization of distributed systems.
- Familiarity with Agile methodology and collaborative team practices.
- Experience with workflow orchestration tools (Airflow/Oozie).
- Strong problem-solving, analytical, and communication skills.
Good to Have
- Knowledge of data modeling and data warehousing concepts.
- Exposure to DevOps practices and CI/CD pipelines for data engineering.
- Experience with other Big Data frameworks such as Spark Streaming or Kafka.
Big Data Engineer
Posted 7 days ago
Job Viewed
Job Description
Location: Pune/Nagpur (WFO)
Experience: 8 - 12 Years
Employment Type: Full-time
Job Overview
We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance and reliability.
Key Responsibilities
Design, develop, and maintain data pipelines using PySpark and related Big Data technologies.
Work with HDFS, Hive, Sqoop , and other tools in the Hadoop ecosystem.
Write efficient HiveQL and SQL queries to handle large-scale datasets.
Perform performance tuning and optimization of distributed data systems.
Collaborate with cross-functional teams in an Agile environment to deliver high-quality solutions.
Manage and schedule workflows using Apache Airflow or Oozie .
Troubleshoot and resolve issues in data pipelines to ensure reliability and accuracy.
Required Skills
Proven experience in Big Data Engineering with a focus on PySpark.
Strong knowledge of HDFS, Hive, Sqoop , and related tools.
Proficiency in SQL/HiveQL for large datasets.
Expertise in performance tuning and optimization of distributed systems.
Familiarity with Agile methodology and collaborative team practices.
Experience with workflow orchestration tools ( Airflow/Oozie ).
Strong problem-solving, analytical, and communication skills.
Good to Have
Knowledge of data modeling and data warehousing concepts.
Exposure to DevOps practices and CI/CD pipelines for data engineering.
Experience with other Big Data frameworks such as Spark Streaming or Kafka .
Big Data Engineer
Posted today
Job Viewed
Job Description
Description
Primary Tech skills – AZURE ,Databricks ,Pyspark,SQL Secondary Tech skills – ADF, Synapse Roles:•Experience in Pyspark, SQL, Cloud data warehouse like Azure Synapse, databricks•Experience working with structured and unstructured data•Extensive knowledge of Data Warehousing concepts, strategies, methodologies•Experience in ETL/Pipeline Development using tools such as Azure Databricks and Azure Data Factory with development expertise on batch and real-time data integration•Experience in data ingestion, preparation, integration, and operationalization techniques in optimally addressing the data requirements•Experience in relational data processing technology like MS SQL, Delta Lake, Spark SQL, SQL Server•Experience with Orchestration tools, Azure DevOps, and GitHub•Must be team oriented with strong communication, collaboration, prioritization, and adaptability skillBig Data Engineer
Posted today
Job Viewed
Job Description
We are looking for passionate B.Tech freshers with strong programming skills in Java who are eager to start their career in Big Data technologies . The role offers exciting opportunities to work on real-time big data projects, data pipelines, and cloud-based data solutions.
Requirements
Assist in designing, developing, and maintaining big data solutions .
Write efficient code in Java and integrate with big data frameworks.
Support in building data ingestion, transformation, and processing pipelines .
Work with distributed systems and learn technologies like Hadoop, Spark, Kafka, Hive, HBase .
Collaborate with senior engineers on data-related problem-solving and performance optimization.
Participate in debugging, testing, and documentation of big data workflows.
Strong knowledge of Core Java & OOPs concepts .
Good understanding of SQL and database concepts .
Familiarity with data structures & algorithms .
Basic knowledge of Big Data frameworks (Hadoop/Spark/Kafka) is an added advantage.
Problem-solving skills and eagerness to learn new technologies.
Education: B.Tech (CSE/IT or related fields).
Batch: (specific, e.g., 2024/2025 pass outs).
Experience: Fresher (0–1 year)
Benefits
Training and mentoring in cutting-edge Big Data tools & technologies .
Exposure to live projects from day one.
A fast-paced, learning-oriented work culture.
Requirements
Strong knowledge of Core Java & OOPs concepts. Good understanding of SQL and database concepts. Familiarity with data structures & algorithms.
Big Data Engineer
Posted today
Job Viewed
Job Description
What You’ll Do:
You are an experienced Software Engineer, Data/Data Engineer who’s passionate about developing products that are simple, intuitive, and beautiful. You will be making an impact on HG’s core data platform/processing engine using modern Big Data techniques particularly working with enterprise applications for B2B. We are a small engineering focused team who are delivering innovative features in a fast-growing market. We build Big Data systems utilizing cutting edge technologies and solutions that allow our developers to learn and develop while shipping amazing products.
You will be involved in building data pipelines at a large scale to enable business teams to work with data and analyze metrics that support and drive the business. You will partner with cross functional teams to identify opportunities and continuously develop and improve processes for efficiency.
What You'll Be Responsible For:
Be The First To Know
About the latest Senior data engineer Jobs in India !
Big Data Engineer
Posted today
Job Viewed
Job Description
Role : Big Data Developer (Data Engineer)
Role OverviewThe role is within the DBI organization (Data Scientist & Data Engineering team) . The Data Engineer will focus on data ingestion, transformation, and processing on Azure Cloud with Databricks & DBT as core tools.
Key Responsibilities-
Analyze data models and provide logical conclusions.
-
Perform process modeling for data workflows.
-
Develop & monitor Azure Cloud components for ingestion, transformation & processing.
-
Troubleshoot and resolve complex data infrastructure issues , including performance tuning.
-
Design and develop programs as per functional & non-functional requirements .
-
Maintain technical documentation and follow change/configuration processes.
2+ years of experience as a Data Engineer / Big Data Developer .
Strong SQL query writing & knowledge of design patterns.
Good understanding of distributed computation .
Hands-on with DBT , Git , Azure Cloud , and Databricks .
VDart: Digital Consulting & Staffing Solutions
VDart is a leading digital consulting and staffing company founded in 2007 and headquartered in Alpharetta, Georgia. VDart is one of the top staffing companies in USA & also provides technology solutions, supporting businesses in their digital transformation journey. Digital consulting services, Digital consulting agency
Core Services:
- Digital consulting and transformation solutions
- Comprehensive staffing services (contract, temporary, permanent placements)
- Talent management and workforce solutions
- Technology implementation services
Key Industries: VDart primarily serves industries such as automotive and mobility, banking and finance, healthcare and life sciences, and energy and utilities. VDart - Products, Competitors, Financials, Employees, Headquarters Locations
Notable Partnership: VDart Digital has been a consistently strong supplier for Toyota since 2017 Digital Transformation Consulting & Strategy Services | VDart Digital , demonstrating their capability in large-scale digital transformation projects.
With over 17 years of experience, VDart combines staffing expertise with technology solutions to help organizations modernize their operations and build skilled workforce capabilities across multiple sectors.
Big Data Engineer
Posted today
Job Viewed
Job Description
Description
Data Engineer: Primary : ADF, SQLSeconday : Databricks, Pyspark / Scala, CICD, Data Security, encryption
Responsibilities: •Work closely with scrum master, technical leads and provide daily progress updates on the tasks assigned. •Provide support in defining the scope and sizing of the work. •Working closely with various enterprise architects, Information security teams, Data management team, to ensure the architected solution meets all the needs of a customer, from a functionality perspective and IT solution engineering perspective. •Translate business requirements into technology solutions. •Lead designing of all aspects of our data solution including artifact creation such as diagrams, playbooks, and other technical documents Skills: •Over all 5 to 8 years of experience with Data Management & Big Data and Analytics. •At least 2 to 3 years of experience in implementing the data solutions on MS Azure using Azure Data Factory. •Must have strong SQL background and ability to tune the SQL for optimized ETL jobs. •one to two years of experience in Azure Synapse Analytics is plus. •Extensive hands-on experience in data warehousing design, tuning and ETL/ELT process development by using cloud native technologies. •Apache Spark experience using Scala or PySpark or pre-packaged tools like Databrick is plus. •At least one year of experience with API based/real time integrations using Kafka or similar technologies. •At least 1 to 2 years of experience with big data using spark and java for data processing. •Developing the CICD pipeline for Azure Infrastructure, version control strategy and Integrate source control (git, GitHub, azure repos). •At least one year experience with unified data governance solution using MS Purview. •In-depth understanding of various storage services offered by Azure. •Experience with implementation of data security, encryption, PII/PSI legislation, identity and access management across sources and environments. •Experience with data process Orchestration, end-to-end design and build process of Near-Real Time and Batch Data Pipelines. •Certification in Azure fundamentals (AZ-900) and Azure data engineering (DP-203) is must. •Strong client-facing communication and facilitation skills.Big Data Engineer
Posted today
Job Viewed
Job Description
Hiring for Big Data:
LOCATION: Chennai, Bengaluru, Hyderabad.
EXPERIENCE: 7-10
Notice Period : Immediate Joiner or 30 Day
Key Skills
- Hands-on experience on technologies like Python, SQL, Snowflake, HDFS , Hive, Scala, Spark, AWS, HBase and Cassandra.
- Good knowledge in Data Warehousing concepts.
- Proficient in Hadoop distributions such as Cloudera, Hortonworks.
- Good working experience on technologies like Python , Scala, SQL & PL/SQL
- Developers design and build the foundational architecture to manage massive-scale data storage, processing, and analysis using distributed, cloud-based systems and platforms.
- Coding Big Data Pipelines.
- Managing Big Data Infrastructure and Pipelines.