512 Data Scientists jobs in Noida
Big Data
Posted today
Job Viewed
Job Description
- Minimum EAP experience - 7 years.- Responsibilities for Big Data Engineer- Perform data cleaning, integration, validation and analysis
- Extensive experience of developing(must), deploying (must) and maintaining(must) the Big Data Ecosystem
- Create and maintain optimal data pipeline architecture
- Assemble large, complex data sets that meet functional / non-functional business requirements
- Create complex data processing jobs in PySpark to load data from RDBMS (Oracle) & process based on complex business rules (must)
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues (must) and support their data infrastructure needs.
- Qualifications for Big Data Engineer:
- Strong track record of building and optimizing ‘big data’ data pipelines, architectures and data sets.
- Working SQL knowledge and experience working with relational databases
- Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
- Strong analytic skills related to working with structured datasets.
- Strong track record of building in build processes supporting data transformation, data structures, metadata, dependency and workload management.
- Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores (optional)
- Experience in supporting and working with cross-functional teams in a dynamic environment.
- Tech Stack
- Experience with big data tools:
- Hadoop Ecosytem (must),Hive (must)
- Impala (must)
- Sqoop (must)
- Python (must)
- Spark with Python (must)
- PySPARK (must) - experience must in Spark Tuning as well
- SPARK-SQL (must)- Any one RDBMS (good to have)
- Any cloud platform (S3 or Azure)
- Rich experience in scripting language Python (must)
- Rich experience in building optimal pySpark jobs handling huge data load (must)
- Rich experience in performing capacity management and gauging memory needs for huge data processing (must)
- Experience with stream-processing systems: Storm, Spark-Streaming, etc. (good to have)
**Job Summary**:
**Role Based Competencies**:
- Tech - Requirement Management S/W- Tech - NFR- Tech - High level design- Tech - Code quality and coding standards- Beh - Result Orientation- Tech - Code Reviews- Tech - Build Management- Tech - Unit Testing- Beh - Information Seeking- Tech - Agile Methodology- Tech - Analytical Problem Solving- Beh - Communication- Beh - Customer Orientation- Beh - Collaboration**Mandatory Competencies**:
- Big Data - Hadoop- Big Data - Hive- Big Data - Impala- Big Data - PySpark**Good to Have Competencies**:
Big Data Developer
Posted 4 days ago
Job Viewed
Job Description
Position: Big Data Engineer
Experience: 4+ years
Location: All India-Remote, Hyderabad- Hybrid
Notice Period: Immediate/7 days joiners mandate
Job Overview:
Must have skills- Big Data, Scala, AWS and Python or Java
Big Data Developer
Posted 4 days ago
Job Viewed
Job Description
Must have Skills:
Kotlin/Scala/Java
Spark
SQL
Spark Streaming
Any cloud (AWS preferable)
Kafka /Kinesis/Any streaming services
Object-Oriented Programming
Hive, ETL/ELT design experience
CICD experience (ETL pipeline deployment)
Data Modeling experience
Good to Have Skills:
Git/similar version control tool
Knowledge in CI/CD, Microservices
Role Objective:
Big Data Engineer will be responsible for expanding and optimizing our data and database architecture, as well as optimizing data flow and collection for cross functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building. The Data Engineer will support our software developers, database architects, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. They must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products
Roles & Responsibilities:
Sound knowledge in Spark architecture and distributed computing and Spark streaming.
Proficient in Spark – including RDD and Data frames core functions, troubleshooting and performance tuning.
Good understanding in object-oriented concepts and hands on experience on Kotlin/Scala/Java with excellent programming logic and technique.
Good in functional programming and OOPS concept on Kotlin/Scala/Java
Good experience in SQL
Managing the team of Associates and Senior Associates and ensuring the utilization is maintained across the project.
Able to mentor new members for onboarding to the project.
Understand the client requirement and able to design, develop from scratch and deliver.
AWS cloud experience would be preferable.
Experience in analyzing, re-architecting, and re-platforming on-premises data warehouses to data platforms on cloud (AWS is preferred)
Leading the client calls to flag off any delays, blockers, escalations and collate all the requirements.
Managing project timing, client expectations and meeting deadlines.
Should have played project and team management roles.
Facilitate meetings within the team on regular basis.
Understand business requirement and analyze different approaches and plan deliverables and milestones for the project.
Optimization, maintenance, and support of pipelines.
Strong analytical and logical skills.
Ability to comfortably tackling new challenges and learn
Big Data Developer
Posted today
Job Viewed
Job Description
Experience: 5 to 9 years
Must have Skills:
- Kotlin/Scala/Java
- Spark
- SQL
- Spark Streaming
- Any cloud (AWS preferable)
- Kafka /Kinesis/Any streaming services
- Object-Oriented Programming
- Hive, ETL/ELT design experience
- CICD experience (ETL pipeline deployment)
- Data Modeling experience
Good to Have Skills:
- Git/similar version control tool
- Knowledge in CI/CD, Microservices
Role Objective:
Big Data Engineer will be responsible for expanding and optimizing our data and database architecture, as well as optimizing data flow and collection for cross functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building. The Data Engineer will support our software developers, database architects, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. They must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products
Roles & Responsibilities:
- Sound knowledge in Spark architecture and distributed computing and Spark streaming.
- Proficient in Spark – including RDD and Data frames core functions, troubleshooting and performance tuning.
- Good understanding in object-oriented concepts and hands on experience on Kotlin/Scala/Java with excellent programming logic and technique.
- Good in functional programming and OOPS concept on Kotlin/Scala/Java
- Good experience in SQL
- Managing the team of Associates and Senior Associates and ensuring the utilization is maintained across the project.
- Able to mentor new members for onboarding to the project.
- Understand the client requirement and able to design, develop from scratch and deliver.
- AWS cloud experience would be preferable.
- Experience in analyzing, re-architecting, and re-platforming on-premises data warehouses to data platforms on cloud (AWS is preferred)
- Leading the client calls to flag off any delays, blockers, escalations and collate all the requirements.
- Managing project timing, client expectations and meeting deadlines.
- Should have played project and team management roles.
- Facilitate meetings within the team on regular basis.
- Understand business requirement and analyze different approaches and plan deliverables and milestones for the project.
- Optimization, maintenance, and support of pipelines.
- Strong analytical and logical skills.
- Ability to comfortably tackling new challenges and learn
Big Data Developer
Posted today
Job Viewed
Job Description
Role Highlights:
Position: Big Data Engineer
Experience: 4+ years
Location: All India-Remote, Hyderabad- Hybrid
Notice Period: Immediate/7 days joiners mandate
Job Overview:
Must have skills- Big Data, Scala, AWS and Python or Java
Big Data Engineer
Posted today
Job Viewed
Job Description
We are looking for passionate B.Tech freshers with strong programming skills in Java who are eager to start their career in Big Data technologies . The role offers exciting opportunities to work on real-time big data projects, data pipelines, and cloud-based data solutions.
Requirements
Assist in designing, developing, and maintaining big data solutions .
Write efficient code in Java and integrate with big data frameworks.
Support in building data ingestion, transformation, and processing pipelines .
Work with distributed systems and learn technologies like Hadoop, Spark, Kafka, Hive, HBase .
Collaborate with senior engineers on data-related problem-solving and performance optimization.
Participate in debugging, testing, and documentation of big data workflows.
Strong knowledge of Core Java & OOPs concepts .
Good understanding of SQL and database concepts .
Familiarity with data structures & algorithms .
Basic knowledge of Big Data frameworks (Hadoop/Spark/Kafka) is an added advantage.
Problem-solving skills and eagerness to learn new technologies.
Education: B.Tech (CSE/IT or related fields).
Batch: (specific, e.g., 2024/2025 pass outs).
Experience: Fresher (0–1 year)
Benefits
Training and mentoring in cutting-edge Big Data tools & technologies .
Exposure to live projects from day one.
A fast-paced, learning-oriented work culture.
Big Data Engineer
Posted today
Job Viewed
Job Description
We are looking for passionate B.Tech freshers with strong programming skills in Java who are eager to start their career in Big Data technologies . The role offers exciting opportunities to work on real-time big data projects, data pipelines, and cloud-based data solutions.
Requirements
Assist in designing, developing, and maintaining big data solutions .
Write efficient code in Java and integrate with big data frameworks.
Support in building data ingestion, transformation, and processing pipelines .
Work with distributed systems and learn technologies like Hadoop, Spark, Kafka, Hive, HBase .
Collaborate with senior engineers on data-related problem-solving and performance optimization.
Participate in debugging, testing, and documentation of big data workflows.
Strong knowledge of Core Java & OOPs concepts .
Good understanding of SQL and database concepts .
Familiarity with data structures & algorithms .
Basic knowledge of Big Data frameworks (Hadoop/Spark/Kafka) is an added advantage.
Problem-solving skills and eagerness to learn new technologies.
Education: B.Tech (CSE/IT or related fields).
Batch: (specific, e.g., 2024/2025 pass outs).
Experience: Fresher (0–1 year)
Benefits
Training and mentoring in cutting-edge Big Data tools & technologies .
Exposure to live projects from day one.
A fast-paced, learning-oriented work culture.
Requirements
Strong knowledge of Core Java & OOPs concepts. Good understanding of SQL and database concepts. Familiarity with data structures & algorithms.
Be The First To Know
About the latest Data scientists Jobs in Noida !
Big Data Engineer
Posted today
Job Viewed
Job Description
Basic Qualifications:
Bachelors degree or higher in Computer Science, or equivalent degree and 3-10 years related working experience.
In-depth experience with a big data cloud platform, preferably Azure.
Strong grasp of programming languages (Python, PySpark, or equivalent) and a willingness to learn new ones.
Experience writing database-heavy services or APIs.
Experience building and optimizing data pipelines, architectures, and data sets.
Working knowledge of queueing, stream processing, and highly scalable data stores
Experience working with and supporting cross-functional teams.
Strong understanding of structuring code for testability.
Preferred Qualifications:
Professional experience implementing and maintaining MLOps pipelines in MLflow or AzureML.
Professional experience implementing data ingestion pipelines using Data Factory.
Professional experience with Databricks and coding with notebooks.
Professional experience processing and manipulating data using SQL and Python code.
Professional experience with user training, customer support, and coordination with cross-functional teams.
Big Data Engineer
Posted today
Job Viewed
Job Description
We are seeking a skilled Big Data Engineer to join our team in India. The ideal candidate will have a strong background in designing and implementing data architectures and pipelines, and will be responsible for ensuring the efficient processing and storage of large datasets.
Responsibilities- Design and implement scalable data pipelines to support data ingestion, processing, and storage.
- Collaborate with data scientists and analysts to understand data requirements and provide necessary data solutions.
- Optimize and maintain existing data architectures to ensure high performance and reliability.
- Develop and maintain documentation for data engineering processes and data flow diagrams.
- Monitor and troubleshoot data pipeline issues to ensure data integrity and availability.
- 5-10 years of experience in Big Data technologies such as Hadoop, Spark, and Kafka.
- Proficient in programming languages such as Java, Python, or Scala.
- Experience with data modeling and database design, including both SQL and NoSQL databases.
- Strong understanding of ETL processes and tools.
- Familiarity with cloud platforms such as AWS, Azure, or Google Cloud.
- Knowledge of data warehousing solutions and architectures.
- Experience with containerization technologies like Docker and orchestration tools like Kubernetes.
Education
Bachelor Of Computer Application (B.C.A), Bachelor Of Technology (B.Tech/B.E), Master in Computer Application (M.C.A), Post Graduate Diploma in Computer Applications (PGDCA), Masters in Technology (M.Tech/M.E)
Skills Required
Hadoop, Spark, Kafka, Sql, Nosql, Python, Data Warehousing, Etl, Data Modeling, Cloud Services
Big Data Developer
Posted 2 days ago
Job Viewed
Job Description
Experience: 5 to 9 years
Must have Skills:
- Kotlin/Scala/Java
- Spark
- SQL
- Spark Streaming
- Any cloud (AWS preferable)
- Kafka /Kinesis/Any streaming services
- Object-Oriented Programming
- Hive, ETL/ELT design experience
- CICD experience (ETL pipeline deployment)
- Data Modeling experience
Good to Have Skills:
- Git/similar version control tool
- Knowledge in CI/CD, Microservices
Role Objective:
Big Data Engineer will be responsible for expanding and optimizing our data and database architecture, as well as optimizing data flow and collection for cross functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building. The Data Engineer will support our software developers, database architects, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. They must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products
Roles & Responsibilities:
- Sound knowledge in Spark architecture and distributed computing and Spark streaming.
- Proficient in Spark – including RDD and Data frames core functions, troubleshooting and performance tuning.
- Good understanding in object-oriented concepts and hands on experience on Kotlin/Scala/Java with excellent programming logic and technique.
- Good in functional programming and OOPS concept on Kotlin/Scala/Java
- Good experience in SQL
- Managing the team of Associates and Senior Associates and ensuring the utilization is maintained across the project.
- Able to mentor new members for onboarding to the project.
- Understand the client requirement and able to design, develop from scratch and deliver.
- AWS cloud experience would be preferable.
- Experience in analyzing, re-architecting, and re-platforming on-premises data warehouses to data platforms on cloud (AWS is preferred)
- Leading the client calls to flag off any delays, blockers, escalations and collate all the requirements.
- Managing project timing, client expectations and meeting deadlines.
- Should have played project and team management roles.
- Facilitate meetings within the team on regular basis.
- Understand business requirement and analyze different approaches and plan deliverables and milestones for the project.
- Optimization, maintenance, and support of pipelines.
- Strong analytical and logical skills.
- Ability to comfortably tackling new challenges and learn