Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T04:09:19Z

Kgala2: /* Test Plan */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
[[File:f4.png|1000px]]
#Nutanix NDB is a distributed NoSQL database service that is part of the Nutanix platform. Some of the key features of NDB include highly scalable architecture, distributed data storage, support for multiple data models, consistent data, fast data access, automatic sharding, real-time analytics, high availability and fault tolerance, and strong security features.
#With its ability to scale up or down the number of nodes in a cluster, Nutanix NDB provides highly scalable architecture without any downtime. Its distributed architecture ensures high availability and fault tolerance, while its support for multiple data models makes it a versatile database service for a wide range of use cases. Additionally, NDB supports strong consistency and fast data access by caching frequently accessed data in memory, which helps reduce the number of disk reads and improves query performance.
#NDB also provides automatic sharding, which helps ensure that your database can handle large amounts of data. You can use graph queries to analyze relationships between data in real-time, which can help you make more informed decisions. Furthermore, NDB offers high availability and fault tolerance through its distributed architecture and replication features. Lastly, NDB provides strong security features, including role-based access control, data encryption at rest, and network security features.

[[File:f1.jpg|1200px]]

===NDB Kubernetes Operator===

The NDB Kubernetes Operator is an innovative tool created by Nutanix to streamline the management and operation of the Nutanix NDB (NoSQL database) on Kubernetes clusters.

With the NDB Kubernetes Operator, deploying and managing NDB clusters on Kubernetes has never been easier, as it eliminates the need to manually configure and manage the underlying infrastructure. Built on the Kubernetes operator framework, it offers a declarative way to manage the lifecycle of NDB clusters and other related resources.

One of the key benefits of the operator is that it simplifies the management of NDB clusters by automating common tasks, such as cluster creation, scaling, upgrading, backup, and recovery. It also offers a high degree of flexibility and customization, allowing you to configure various aspects of the cluster, such as storage, networking, and security.

Another advantage of the NDB Kubernetes Operator is its seamless integration with other Kubernetes tools and resources, such as Helm charts, Kubernetes secrets, and Kubernetes ConfigMaps. This integration makes it easy to integrate NDB into your existing Kubernetes-based infrastructure and workflows, providing a hassle-free solution for managing your database clusters.

Overall, the NDB Kubernetes Operator is a powerful and flexible tool for managing NDB clusters on Kubernetes, freeing you up to focus on your application logic rather than infrastructure management. Its automation capabilities and integration with other Kubernetes tools make it a must-have tool for developers and administrators looking to simplify and streamline their database management on Kubernetes.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:file2.png|1300px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
Large amounts of data may be handled by the highly scalable, fault-tolerant, and consistent Nutanix NDB NoSQL database. It is a distributed database created to be installed over several cluster nodes. A portion of the data is stored on each node in the cluster, and the data is replicated across several nodes to guarantee high availability.

Configure your Nutanix cluster: We need to configure your Nutanix cluster to support NDB. This includes setting up the storage and network configurations, configuring the NDB nodes, and defining the replication factor.

Create a table: We need to create a table in NDB to store your data. This includes defining the schema, specifying the replication factor, and configuring any other options you need.

Write your code: We need to write your code to interact with the NDB cluster. This includes inserting and retrieving data, as well as performing more complex operations such as querying, indexing, and data aggregation.

Test your code: We need to test your code to ensure that it works as expected. This includes testing basic operations such as creating and retrieving data, as well as testing more complex operations such as queries and data aggregation.

Monitor your cluster: We need to monitor your NDB cluster to ensure that it is performing as expected. This includes monitoring resource usage, handling errors and exceptions, and optimizing performance.

Optimize your cluster: We need to optimize your NDB cluster over time to ensure that it continues to meet your needs. This includes tuning the configuration, optimizing queries, and scaling the cluster as needed.

Backup and recovery: We need to establish backup and recovery procedures to ensure that your data is protected against data loss or corruption. This includes regularly backing up your data, testing your backups, and establishing procedures for recovering data in case of a disaster.

[[File:file5.png|1000px]]

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' This pattern could be used to create the provisioning request for a database instance in a more modular and flexible way. Rather than creating the request directly in one function, a builder class could be used to set individual properties of the request. This would make the code more maintainable and extensible, and would allow for easier testing of different combinations of request properties.

'''Factory:''' Since there are different types of database instances that can be provisioned (e.g. MySQL, Postgres, etc.), a factory pattern could be used to create the appropriate request object based on the specified database type. This would help to decouple the creation of the request object from the calling code, and would make it easier to add support for new database types in the future.

'''Dependency Injection:''' To allow for better testability, dependency injection can be used to decouple the code from its dependencies. For example, in the provided code snippet, the NDBClient is being passed into the GenerateProvisioningRequest() function. However, if the NDBClient had additional dependencies or if it were difficult to create a testable version of the NDBClient, dependency injection could be used to allow for easier testing and swapping of dependencies.

''' Code Refactoring:'''
After reviewing the code base, it was discovered that the ndb_api_helpers.go file contains the code for provisioning the database. The main task of generating the request payload for provisioning the database is handled by the GenerateProvisioningRequest function. To retrieve all the profiles, this function utilizes the GetOOBProfiles function which returns a map of all the profiles. However, the current implementation of GenerateProvisioningRequest only retrieves the first element of the values within the map, which is assumed to be the default value. This means that the function doesn't verify if the user has provided a specific profile or not before assigning a default value.

To improve this behavior, we plan to iterate over all the profiles in the arrays that are inside the values of the profiles map. If the user has provided input for a specific profile, we will assign that input to the profiles variable. If not, we will use the first element of the array as the default value.

This change will allow us to properly check whether the user has provided input for a specific profile or not, and avoid the incorrect assumption that the first element of the values array is always the default value. By iterating over all the profiles, we can ensure that the correct profile is selected and assigned to the profiles variable. This will lead to more accurate and reliable database provisioning.

[[File:file6.png|1000px]]

[[File:file7.png|1000px]]

== Test Plan ==

=== Test Case Scenario 1 ===

Test case name: Provisioning of appropriate database based on compute parameters
** Description: This test case verifies that the appropriate database is provisioned based on the compute parameters passed as input, as expected.
** Pre-conditions:
*** The system is set up and running
*** The compute parameters are available for input
** Test steps:
*** Pass the compute parameters as input to the system
*** Check if the appropriate database has been provisioned based on the compute parameters
*** Verify that the database configuration and settings match the expected values based on the input parameters
** Expected results:
*** The system provisions the appropriate database based on the compute parameters
*** The database configuration and settings match the expected values based on the input parameters
*** The test case passes successfully
=== Test Case Scenario 2 ===

Test case name: Use of default setup for configuration when optional parameters are not passed
** Description: This test case verifies that the database configured uses the default setup for configuration when optional parameters are not passed as input, as expected.
** Pre-conditions:
*** The system is set up and running
*** No optional parameters are passed as input
** Test steps:
*** Do not pass any optional parameters as input to the system
*** Check if the database is configured using the default setup for configuration
*** Verify that the database configuration and settings match the expected values based on the default setup
** Expected results:
*** The system configures the database using the default setup when no optional parameters are passed
*** The database configuration and settings match the expected values based on the default setup
*** The test case passes successfully

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

[3] NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/

[4] Go Operator SDK - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T03:57:32Z

Kgala2: /* References */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
[[File:f4.png|1000px]]
#Nutanix NDB is a distributed NoSQL database service that is part of the Nutanix platform. Some of the key features of NDB include highly scalable architecture, distributed data storage, support for multiple data models, consistent data, fast data access, automatic sharding, real-time analytics, high availability and fault tolerance, and strong security features.
#With its ability to scale up or down the number of nodes in a cluster, Nutanix NDB provides highly scalable architecture without any downtime. Its distributed architecture ensures high availability and fault tolerance, while its support for multiple data models makes it a versatile database service for a wide range of use cases. Additionally, NDB supports strong consistency and fast data access by caching frequently accessed data in memory, which helps reduce the number of disk reads and improves query performance.
#NDB also provides automatic sharding, which helps ensure that your database can handle large amounts of data. You can use graph queries to analyze relationships between data in real-time, which can help you make more informed decisions. Furthermore, NDB offers high availability and fault tolerance through its distributed architecture and replication features. Lastly, NDB provides strong security features, including role-based access control, data encryption at rest, and network security features.

[[File:f1.jpg|1200px]]

===NDB Kubernetes Operator===

The NDB Kubernetes Operator is an innovative tool created by Nutanix to streamline the management and operation of the Nutanix NDB (NoSQL database) on Kubernetes clusters.

With the NDB Kubernetes Operator, deploying and managing NDB clusters on Kubernetes has never been easier, as it eliminates the need to manually configure and manage the underlying infrastructure. Built on the Kubernetes operator framework, it offers a declarative way to manage the lifecycle of NDB clusters and other related resources.

One of the key benefits of the operator is that it simplifies the management of NDB clusters by automating common tasks, such as cluster creation, scaling, upgrading, backup, and recovery. It also offers a high degree of flexibility and customization, allowing you to configure various aspects of the cluster, such as storage, networking, and security.

Another advantage of the NDB Kubernetes Operator is its seamless integration with other Kubernetes tools and resources, such as Helm charts, Kubernetes secrets, and Kubernetes ConfigMaps. This integration makes it easy to integrate NDB into your existing Kubernetes-based infrastructure and workflows, providing a hassle-free solution for managing your database clusters.

Overall, the NDB Kubernetes Operator is a powerful and flexible tool for managing NDB clusters on Kubernetes, freeing you up to focus on your application logic rather than infrastructure management. Its automation capabilities and integration with other Kubernetes tools make it a must-have tool for developers and administrators looking to simplify and streamline their database management on Kubernetes.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:file2.png|1300px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
Large amounts of data may be handled by the highly scalable, fault-tolerant, and consistent Nutanix NDB NoSQL database. It is a distributed database created to be installed over several cluster nodes. A portion of the data is stored on each node in the cluster, and the data is replicated across several nodes to guarantee high availability.

Configure your Nutanix cluster: We need to configure your Nutanix cluster to support NDB. This includes setting up the storage and network configurations, configuring the NDB nodes, and defining the replication factor.

Create a table: We need to create a table in NDB to store your data. This includes defining the schema, specifying the replication factor, and configuring any other options you need.

Write your code: We need to write your code to interact with the NDB cluster. This includes inserting and retrieving data, as well as performing more complex operations such as querying, indexing, and data aggregation.

Test your code: We need to test your code to ensure that it works as expected. This includes testing basic operations such as creating and retrieving data, as well as testing more complex operations such as queries and data aggregation.

Monitor your cluster: We need to monitor your NDB cluster to ensure that it is performing as expected. This includes monitoring resource usage, handling errors and exceptions, and optimizing performance.

Optimize your cluster: We need to optimize your NDB cluster over time to ensure that it continues to meet your needs. This includes tuning the configuration, optimizing queries, and scaling the cluster as needed.

Backup and recovery: We need to establish backup and recovery procedures to ensure that your data is protected against data loss or corruption. This includes regularly backing up your data, testing your backups, and establishing procedures for recovering data in case of a disaster.

[[File:file5.png|1000px]]

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' This pattern could be used to create the provisioning request for a database instance in a more modular and flexible way. Rather than creating the request directly in one function, a builder class could be used to set individual properties of the request. This would make the code more maintainable and extensible, and would allow for easier testing of different combinations of request properties.

'''Factory:''' Since there are different types of database instances that can be provisioned (e.g. MySQL, Postgres, etc.), a factory pattern could be used to create the appropriate request object based on the specified database type. This would help to decouple the creation of the request object from the calling code, and would make it easier to add support for new database types in the future.

'''Dependency Injection:''' To allow for better testability, dependency injection can be used to decouple the code from its dependencies. For example, in the provided code snippet, the NDBClient is being passed into the GenerateProvisioningRequest() function. However, if the NDBClient had additional dependencies or if it were difficult to create a testable version of the NDBClient, dependency injection could be used to allow for easier testing and swapping of dependencies.

''' Code Refactoring:'''
After reviewing the code base, it was discovered that the ndb_api_helpers.go file contains the code for provisioning the database. The main task of generating the request payload for provisioning the database is handled by the GenerateProvisioningRequest function. To retrieve all the profiles, this function utilizes the GetOOBProfiles function which returns a map of all the profiles. However, the current implementation of GenerateProvisioningRequest only retrieves the first element of the values within the map, which is assumed to be the default value. This means that the function doesn't verify if the user has provided a specific profile or not before assigning a default value.

To improve this behavior, we plan to iterate over all the profiles in the arrays that are inside the values of the profiles map. If the user has provided input for a specific profile, we will assign that input to the profiles variable. If not, we will use the first element of the array as the default value.

This change will allow us to properly check whether the user has provided input for a specific profile or not, and avoid the incorrect assumption that the first element of the values array is always the default value. By iterating over all the profiles, we can ensure that the correct profile is selected and assigned to the profiles variable. This will lead to more accurate and reliable database provisioning.

[[File:file6.png|1000px]]

[[File:file7.png|1000px]]

==Test Plan==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

[3] NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/

[4] Go Operator SDK - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T03:46:01Z

Kgala2:

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
[[File:f4.png|1000px]]
#Nutanix NDB is a distributed NoSQL database service that is part of the Nutanix platform. Some of the key features of NDB include highly scalable architecture, distributed data storage, support for multiple data models, consistent data, fast data access, automatic sharding, real-time analytics, high availability and fault tolerance, and strong security features.
#With its ability to scale up or down the number of nodes in a cluster, Nutanix NDB provides highly scalable architecture without any downtime. Its distributed architecture ensures high availability and fault tolerance, while its support for multiple data models makes it a versatile database service for a wide range of use cases. Additionally, NDB supports strong consistency and fast data access by caching frequently accessed data in memory, which helps reduce the number of disk reads and improves query performance.
#NDB also provides automatic sharding, which helps ensure that your database can handle large amounts of data. You can use graph queries to analyze relationships between data in real-time, which can help you make more informed decisions. Furthermore, NDB offers high availability and fault tolerance through its distributed architecture and replication features. Lastly, NDB provides strong security features, including role-based access control, data encryption at rest, and network security features.

[[File:f1.jpg|1200px]]

===NDB Kubernetes Operator===

The NDB Kubernetes Operator is an innovative tool created by Nutanix to streamline the management and operation of the Nutanix NDB (NoSQL database) on Kubernetes clusters.

With the NDB Kubernetes Operator, deploying and managing NDB clusters on Kubernetes has never been easier, as it eliminates the need to manually configure and manage the underlying infrastructure. Built on the Kubernetes operator framework, it offers a declarative way to manage the lifecycle of NDB clusters and other related resources.

One of the key benefits of the operator is that it simplifies the management of NDB clusters by automating common tasks, such as cluster creation, scaling, upgrading, backup, and recovery. It also offers a high degree of flexibility and customization, allowing you to configure various aspects of the cluster, such as storage, networking, and security.

Another advantage of the NDB Kubernetes Operator is its seamless integration with other Kubernetes tools and resources, such as Helm charts, Kubernetes secrets, and Kubernetes ConfigMaps. This integration makes it easy to integrate NDB into your existing Kubernetes-based infrastructure and workflows, providing a hassle-free solution for managing your database clusters.

Overall, the NDB Kubernetes Operator is a powerful and flexible tool for managing NDB clusters on Kubernetes, freeing you up to focus on your application logic rather than infrastructure management. Its automation capabilities and integration with other Kubernetes tools make it a must-have tool for developers and administrators looking to simplify and streamline their database management on Kubernetes.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:file2.png|1300px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
Large amounts of data may be handled by the highly scalable, fault-tolerant, and consistent Nutanix NDB NoSQL database. It is a distributed database created to be installed over several cluster nodes. A portion of the data is stored on each node in the cluster, and the data is replicated across several nodes to guarantee high availability.

Configure your Nutanix cluster: We need to configure your Nutanix cluster to support NDB. This includes setting up the storage and network configurations, configuring the NDB nodes, and defining the replication factor.

Create a table: We need to create a table in NDB to store your data. This includes defining the schema, specifying the replication factor, and configuring any other options you need.

Write your code: We need to write your code to interact with the NDB cluster. This includes inserting and retrieving data, as well as performing more complex operations such as querying, indexing, and data aggregation.

Test your code: We need to test your code to ensure that it works as expected. This includes testing basic operations such as creating and retrieving data, as well as testing more complex operations such as queries and data aggregation.

Monitor your cluster: We need to monitor your NDB cluster to ensure that it is performing as expected. This includes monitoring resource usage, handling errors and exceptions, and optimizing performance.

Optimize your cluster: We need to optimize your NDB cluster over time to ensure that it continues to meet your needs. This includes tuning the configuration, optimizing queries, and scaling the cluster as needed.

Backup and recovery: We need to establish backup and recovery procedures to ensure that your data is protected against data loss or corruption. This includes regularly backing up your data, testing your backups, and establishing procedures for recovering data in case of a disaster.

[[File:file5.png|1000px]]

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' This pattern could be used to create the provisioning request for a database instance in a more modular and flexible way. Rather than creating the request directly in one function, a builder class could be used to set individual properties of the request. This would make the code more maintainable and extensible, and would allow for easier testing of different combinations of request properties.

'''Factory:''' Since there are different types of database instances that can be provisioned (e.g. MySQL, Postgres, etc.), a factory pattern could be used to create the appropriate request object based on the specified database type. This would help to decouple the creation of the request object from the calling code, and would make it easier to add support for new database types in the future.

'''Dependency Injection:''' To allow for better testability, dependency injection can be used to decouple the code from its dependencies. For example, in the provided code snippet, the NDBClient is being passed into the GenerateProvisioningRequest() function. However, if the NDBClient had additional dependencies or if it were difficult to create a testable version of the NDBClient, dependency injection could be used to allow for easier testing and swapping of dependencies.

''' Code Refactoring:'''
After reviewing the code base, it was discovered that the ndb_api_helpers.go file contains the code for provisioning the database. The main task of generating the request payload for provisioning the database is handled by the GenerateProvisioningRequest function. To retrieve all the profiles, this function utilizes the GetOOBProfiles function which returns a map of all the profiles. However, the current implementation of GenerateProvisioningRequest only retrieves the first element of the values within the map, which is assumed to be the default value. This means that the function doesn't verify if the user has provided a specific profile or not before assigning a default value.

To improve this behavior, we plan to iterate over all the profiles in the arrays that are inside the values of the profiles map. If the user has provided input for a specific profile, we will assign that input to the profiles variable. If not, we will use the first element of the array as the default value.

This change will allow us to properly check whether the user has provided input for a specific profile or not, and avoid the incorrect assumption that the first element of the values array is always the default value. By iterating over all the profiles, we can ensure that the correct profile is selected and assigned to the profiles variable. This will lead to more accurate and reliable database provisioning.

[[File:file6.png|1000px]]

[[File:file7.png|1000px]]

==Test Plan==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

File:File7.png

2023-03-26T03:44:21Z

Kgala2:

File:File6.png

2023-03-26T03:44:00Z

Kgala2:

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T03:43:51Z

Kgala2: /* Potential Design Patterns, Principles, and Code Refactoring strategies */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
[[File:f4.png|1000px]]
#Nutanix NDB is a distributed NoSQL database service that is part of the Nutanix platform. Some of the key features of NDB include highly scalable architecture, distributed data storage, support for multiple data models, consistent data, fast data access, automatic sharding, real-time analytics, high availability and fault tolerance, and strong security features.
#With its ability to scale up or down the number of nodes in a cluster, Nutanix NDB provides highly scalable architecture without any downtime. Its distributed architecture ensures high availability and fault tolerance, while its support for multiple data models makes it a versatile database service for a wide range of use cases. Additionally, NDB supports strong consistency and fast data access by caching frequently accessed data in memory, which helps reduce the number of disk reads and improves query performance.
#NDB also provides automatic sharding, which helps ensure that your database can handle large amounts of data. You can use graph queries to analyze relationships between data in real-time, which can help you make more informed decisions. Furthermore, NDB offers high availability and fault tolerance through its distributed architecture and replication features. Lastly, NDB provides strong security features, including role-based access control, data encryption at rest, and network security features.

[[File:f1.jpg|1200px]]

===NDB Kubernetes Operator===

The NDB Kubernetes Operator is an innovative tool created by Nutanix to streamline the management and operation of the Nutanix NDB (NoSQL database) on Kubernetes clusters.

With the NDB Kubernetes Operator, deploying and managing NDB clusters on Kubernetes has never been easier, as it eliminates the need to manually configure and manage the underlying infrastructure. Built on the Kubernetes operator framework, it offers a declarative way to manage the lifecycle of NDB clusters and other related resources.

One of the key benefits of the operator is that it simplifies the management of NDB clusters by automating common tasks, such as cluster creation, scaling, upgrading, backup, and recovery. It also offers a high degree of flexibility and customization, allowing you to configure various aspects of the cluster, such as storage, networking, and security.

Another advantage of the NDB Kubernetes Operator is its seamless integration with other Kubernetes tools and resources, such as Helm charts, Kubernetes secrets, and Kubernetes ConfigMaps. This integration makes it easy to integrate NDB into your existing Kubernetes-based infrastructure and workflows, providing a hassle-free solution for managing your database clusters.

Overall, the NDB Kubernetes Operator is a powerful and flexible tool for managing NDB clusters on Kubernetes, freeing you up to focus on your application logic rather than infrastructure management. Its automation capabilities and integration with other Kubernetes tools make it a must-have tool for developers and administrators looking to simplify and streamline their database management on Kubernetes.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:file2.png|1300px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
Large amounts of data may be handled by the highly scalable, fault-tolerant, and consistent Nutanix NDB NoSQL database. It is a distributed database created to be installed over several cluster nodes. A portion of the data is stored on each node in the cluster, and the data is replicated across several nodes to guarantee high availability.

Configure your Nutanix cluster: We need to configure your Nutanix cluster to support NDB. This includes setting up the storage and network configurations, configuring the NDB nodes, and defining the replication factor.

Create a table: We need to create a table in NDB to store your data. This includes defining the schema, specifying the replication factor, and configuring any other options you need.

Write your code: We need to write your code to interact with the NDB cluster. This includes inserting and retrieving data, as well as performing more complex operations such as querying, indexing, and data aggregation.

Test your code: We need to test your code to ensure that it works as expected. This includes testing basic operations such as creating and retrieving data, as well as testing more complex operations such as queries and data aggregation.

Monitor your cluster: We need to monitor your NDB cluster to ensure that it is performing as expected. This includes monitoring resource usage, handling errors and exceptions, and optimizing performance.

Optimize your cluster: We need to optimize your NDB cluster over time to ensure that it continues to meet your needs. This includes tuning the configuration, optimizing queries, and scaling the cluster as needed.

Backup and recovery: We need to establish backup and recovery procedures to ensure that your data is protected against data loss or corruption. This includes regularly backing up your data, testing your backups, and establishing procedures for recovering data in case of a disaster.

[[File:file5.png|1000px]]

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' This pattern could be used to create the provisioning request for a database instance in a more modular and flexible way. Rather than creating the request directly in one function, a builder class could be used to set individual properties of the request. This would make the code more maintainable and extensible, and would allow for easier testing of different combinations of request properties.

'''Factory:''' Since there are different types of database instances that can be provisioned (e.g. MySQL, Postgres, etc.), a factory pattern could be used to create the appropriate request object based on the specified database type. This would help to decouple the creation of the request object from the calling code, and would make it easier to add support for new database types in the future.

'''Dependency Injection:''' To allow for better testability, dependency injection can be used to decouple the code from its dependencies. For example, in the provided code snippet, the NDBClient is being passed into the GenerateProvisioningRequest() function. However, if the NDBClient had additional dependencies or if it were difficult to create a testable version of the NDBClient, dependency injection could be used to allow for easier testing and swapping of dependencies.

''' Code Refactoring:'''
After reviewing the code base, it was discovered that the ndb_api_helpers.go file contains the code for provisioning the database. The main task of generating the request payload for provisioning the database is handled by the GenerateProvisioningRequest function. To retrieve all the profiles, this function utilizes the GetOOBProfiles function which returns a map of all the profiles. However, the current implementation of GenerateProvisioningRequest only retrieves the first element of the values within the map, which is assumed to be the default value. This means that the function doesn't verify if the user has provided a specific profile or not before assigning a default value.

To improve this behavior, we plan to iterate over all the profiles in the arrays that are inside the values of the profiles map. If the user has provided input for a specific profile, we will assign that input to the profiles variable. If not, we will use the first element of the array as the default value.

This change will allow us to properly check whether the user has provided input for a specific profile or not, and avoid the incorrect assumption that the first element of the values array is always the default value. By iterating over all the profiles, we can ensure that the correct profile is selected and assigned to the profiles variable. This will lead to more accurate and reliable database provisioning.

[[File:file6.png|1000px]]

[[File:file7.png|1000px]]

==Demo==

[[File:Image_mongo.png|800px]]

''Here is the demo video for the developed infra:''

https://www.youtube.com/watch?v=lUarCdA8RP0

==Tests==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3
* Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/74
[[File:SS_8.png|800px]]

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T03:29:34Z

Kgala2: /* Potential Design Patterns, Principles, and Code Refactoring strategies */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
[[File:f4.png|1000px]]
#Nutanix NDB is a distributed NoSQL database service that is part of the Nutanix platform. Some of the key features of NDB include highly scalable architecture, distributed data storage, support for multiple data models, consistent data, fast data access, automatic sharding, real-time analytics, high availability and fault tolerance, and strong security features.
#With its ability to scale up or down the number of nodes in a cluster, Nutanix NDB provides highly scalable architecture without any downtime. Its distributed architecture ensures high availability and fault tolerance, while its support for multiple data models makes it a versatile database service for a wide range of use cases. Additionally, NDB supports strong consistency and fast data access by caching frequently accessed data in memory, which helps reduce the number of disk reads and improves query performance.
#NDB also provides automatic sharding, which helps ensure that your database can handle large amounts of data. You can use graph queries to analyze relationships between data in real-time, which can help you make more informed decisions. Furthermore, NDB offers high availability and fault tolerance through its distributed architecture and replication features. Lastly, NDB provides strong security features, including role-based access control, data encryption at rest, and network security features.

[[File:f1.jpg|1200px]]

===NDB Kubernetes Operator===

The NDB Kubernetes Operator is an innovative tool created by Nutanix to streamline the management and operation of the Nutanix NDB (NoSQL database) on Kubernetes clusters.

With the NDB Kubernetes Operator, deploying and managing NDB clusters on Kubernetes has never been easier, as it eliminates the need to manually configure and manage the underlying infrastructure. Built on the Kubernetes operator framework, it offers a declarative way to manage the lifecycle of NDB clusters and other related resources.

One of the key benefits of the operator is that it simplifies the management of NDB clusters by automating common tasks, such as cluster creation, scaling, upgrading, backup, and recovery. It also offers a high degree of flexibility and customization, allowing you to configure various aspects of the cluster, such as storage, networking, and security.

Another advantage of the NDB Kubernetes Operator is its seamless integration with other Kubernetes tools and resources, such as Helm charts, Kubernetes secrets, and Kubernetes ConfigMaps. This integration makes it easy to integrate NDB into your existing Kubernetes-based infrastructure and workflows, providing a hassle-free solution for managing your database clusters.

Overall, the NDB Kubernetes Operator is a powerful and flexible tool for managing NDB clusters on Kubernetes, freeing you up to focus on your application logic rather than infrastructure management. Its automation capabilities and integration with other Kubernetes tools make it a must-have tool for developers and administrators looking to simplify and streamline their database management on Kubernetes.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:file2.png|1300px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
Large amounts of data may be handled by the highly scalable, fault-tolerant, and consistent Nutanix NDB NoSQL database. It is a distributed database created to be installed over several cluster nodes. A portion of the data is stored on each node in the cluster, and the data is replicated across several nodes to guarantee high availability.

Configure your Nutanix cluster: We need to configure your Nutanix cluster to support NDB. This includes setting up the storage and network configurations, configuring the NDB nodes, and defining the replication factor.

Create a table: We need to create a table in NDB to store your data. This includes defining the schema, specifying the replication factor, and configuring any other options you need.

Write your code: We need to write your code to interact with the NDB cluster. This includes inserting and retrieving data, as well as performing more complex operations such as querying, indexing, and data aggregation.

Test your code: We need to test your code to ensure that it works as expected. This includes testing basic operations such as creating and retrieving data, as well as testing more complex operations such as queries and data aggregation.

Monitor your cluster: We need to monitor your NDB cluster to ensure that it is performing as expected. This includes monitoring resource usage, handling errors and exceptions, and optimizing performance.

Optimize your cluster: We need to optimize your NDB cluster over time to ensure that it continues to meet your needs. This includes tuning the configuration, optimizing queries, and scaling the cluster as needed.

Backup and recovery: We need to establish backup and recovery procedures to ensure that your data is protected against data loss or corruption. This includes regularly backing up your data, testing your backups, and establishing procedures for recovering data in case of a disaster.

[[File:file5.png|1000px]]

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' This pattern could be used to create the provisioning request for a database instance in a more modular and flexible way. Rather than creating the request directly in one function, a builder class could be used to set individual properties of the request. This would make the code more maintainable and extensible, and would allow for easier testing of different combinations of request properties.

'''Factory:''' Since there are different types of database instances that can be provisioned (e.g. MySQL, Postgres, etc.), a factory pattern could be used to create the appropriate request object based on the specified database type. This would help to decouple the creation of the request object from the calling code, and would make it easier to add support for new database types in the future.

'''Dependency Injection:''' To allow for better testability, dependency injection can be used to decouple the code from its dependencies. For example, in the provided code snippet, the NDBClient is being passed into the GenerateProvisioningRequest() function. However, if the NDBClient had additional dependencies or if it were difficult to create a testable version of the NDBClient, dependency injection could be used to allow for easier testing and swapping of dependencies.

''' Code Refactoring:'''
After reviewing the code base, it was discovered that the ndb_api_helpers.go file contains the code for provisioning the database. The main task of generating the request payload for provisioning the database is handled by the GenerateProvisioningRequest function. To retrieve all the profiles, this function utilizes the GetOOBProfiles function which returns a map of all the profiles. However, the current implementation of GenerateProvisioningRequest only retrieves the first element of the values within the map, which is assumed to be the default value. This means that the function doesn't verify if the user has provided a specific profile or not before assigning a default value.

To improve this behavior, we plan to iterate over all the profiles in the arrays that are inside the values of the profiles map. If the user has provided input for a specific profile, we will assign that input to the profiles variable. If not, we will use the first element of the array as the default value.

This change will allow us to properly check whether the user has provided input for a specific profile or not, and avoid the incorrect assumption that the first element of the values array is always the default value. By iterating over all the profiles, we can ensure that the correct profile is selected and assigned to the profiles variable. This will lead to more accurate and reliable database provisioning.

==Demo==

[[File:Image_mongo.png|800px]]

''Here is the demo video for the developed infra:''

https://www.youtube.com/watch?v=lUarCdA8RP0

==Tests==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3
* Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/74
[[File:SS_8.png|800px]]

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

File:File5.png

2023-03-26T03:01:23Z

Kgala2:

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T03:01:12Z

Kgala2: /* Design & Workflow */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
[[File:f4.png|1000px]]
#Nutanix NDB is a distributed NoSQL database service that is part of the Nutanix platform. Some of the key features of NDB include highly scalable architecture, distributed data storage, support for multiple data models, consistent data, fast data access, automatic sharding, real-time analytics, high availability and fault tolerance, and strong security features.
#With its ability to scale up or down the number of nodes in a cluster, Nutanix NDB provides highly scalable architecture without any downtime. Its distributed architecture ensures high availability and fault tolerance, while its support for multiple data models makes it a versatile database service for a wide range of use cases. Additionally, NDB supports strong consistency and fast data access by caching frequently accessed data in memory, which helps reduce the number of disk reads and improves query performance.
#NDB also provides automatic sharding, which helps ensure that your database can handle large amounts of data. You can use graph queries to analyze relationships between data in real-time, which can help you make more informed decisions. Furthermore, NDB offers high availability and fault tolerance through its distributed architecture and replication features. Lastly, NDB provides strong security features, including role-based access control, data encryption at rest, and network security features.

[[File:f1.jpg|1200px]]

===NDB Kubernetes Operator===

The NDB Kubernetes Operator is an innovative tool created by Nutanix to streamline the management and operation of the Nutanix NDB (NoSQL database) on Kubernetes clusters.

With the NDB Kubernetes Operator, deploying and managing NDB clusters on Kubernetes has never been easier, as it eliminates the need to manually configure and manage the underlying infrastructure. Built on the Kubernetes operator framework, it offers a declarative way to manage the lifecycle of NDB clusters and other related resources.

One of the key benefits of the operator is that it simplifies the management of NDB clusters by automating common tasks, such as cluster creation, scaling, upgrading, backup, and recovery. It also offers a high degree of flexibility and customization, allowing you to configure various aspects of the cluster, such as storage, networking, and security.

Another advantage of the NDB Kubernetes Operator is its seamless integration with other Kubernetes tools and resources, such as Helm charts, Kubernetes secrets, and Kubernetes ConfigMaps. This integration makes it easy to integrate NDB into your existing Kubernetes-based infrastructure and workflows, providing a hassle-free solution for managing your database clusters.

Overall, the NDB Kubernetes Operator is a powerful and flexible tool for managing NDB clusters on Kubernetes, freeing you up to focus on your application logic rather than infrastructure management. Its automation capabilities and integration with other Kubernetes tools make it a must-have tool for developers and administrators looking to simplify and streamline their database management on Kubernetes.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:file2.png|1300px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
Large amounts of data may be handled by the highly scalable, fault-tolerant, and consistent Nutanix NDB NoSQL database. It is a distributed database created to be installed over several cluster nodes. A portion of the data is stored on each node in the cluster, and the data is replicated across several nodes to guarantee high availability.

Configure your Nutanix cluster: We need to configure your Nutanix cluster to support NDB. This includes setting up the storage and network configurations, configuring the NDB nodes, and defining the replication factor.

Create a table: We need to create a table in NDB to store your data. This includes defining the schema, specifying the replication factor, and configuring any other options you need.

Write your code: We need to write your code to interact with the NDB cluster. This includes inserting and retrieving data, as well as performing more complex operations such as querying, indexing, and data aggregation.

Test your code: We need to test your code to ensure that it works as expected. This includes testing basic operations such as creating and retrieving data, as well as testing more complex operations such as queries and data aggregation.

Monitor your cluster: We need to monitor your NDB cluster to ensure that it is performing as expected. This includes monitoring resource usage, handling errors and exceptions, and optimizing performance.

Optimize your cluster: We need to optimize your NDB cluster over time to ensure that it continues to meet your needs. This includes tuning the configuration, optimizing queries, and scaling the cluster as needed.

Backup and recovery: We need to establish backup and recovery procedures to ensure that your data is protected against data loss or corruption. This includes regularly backing up your data, testing your backups, and establishing procedures for recovering data in case of a disaster.

[[File:file5.png|1000px]]

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' For any objects that are created, we could instantiate using say method chaining rather than initializing everything with a constructor.

'''Factory:''' Instead of using the regular way (such as the ‘new’ keyword) to instantiate an object, a factory method would be used to do the same.
This pattern can be used if we are creating a superclass for provisioning databases, and subclasses for provisioning different kinds of databases (MongoDB, MySQL, etc). This is because if we want to add another kind of database to our project, and we are creating new databases by conditional checking, our code could get messy. Thus, a factory method could instead create objects in a smarter way for the different database classes (or modules in our case).

'''Facade:''' This pattern could be used for masking the complicated provisioning payloads.

'''Open and Closed principle:''' As per this principle, an interface is open for extension but closed for modification. We could have an interface with provisioning and deprovisioning methods, and interfaces of different databases could extend and reimplement those methods.

'''Adapter''' design pattern could also be added to adapt to different databases

'''DRY(Don’t repeat yourself):''' There are a lot of ways in which DRY principles will be applied in our project:
* Reusing the provisioning function
* Using constants instead of variables
* Extracting all common functionality into reusable modules and functions.

==Demo==

[[File:Image_mongo.png|800px]]

''Here is the demo video for the developed infra:''

https://www.youtube.com/watch?v=lUarCdA8RP0

==Tests==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3
* Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/74
[[File:SS_8.png|800px]]

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T02:55:16Z

Kgala2: /* NDB Kubernetes Operator */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
[[File:f4.png|1000px]]
#Nutanix NDB is a distributed NoSQL database service that is part of the Nutanix platform. Some of the key features of NDB include highly scalable architecture, distributed data storage, support for multiple data models, consistent data, fast data access, automatic sharding, real-time analytics, high availability and fault tolerance, and strong security features.
#With its ability to scale up or down the number of nodes in a cluster, Nutanix NDB provides highly scalable architecture without any downtime. Its distributed architecture ensures high availability and fault tolerance, while its support for multiple data models makes it a versatile database service for a wide range of use cases. Additionally, NDB supports strong consistency and fast data access by caching frequently accessed data in memory, which helps reduce the number of disk reads and improves query performance.
#NDB also provides automatic sharding, which helps ensure that your database can handle large amounts of data. You can use graph queries to analyze relationships between data in real-time, which can help you make more informed decisions. Furthermore, NDB offers high availability and fault tolerance through its distributed architecture and replication features. Lastly, NDB provides strong security features, including role-based access control, data encryption at rest, and network security features.

[[File:f1.jpg|1200px]]

===NDB Kubernetes Operator===

The NDB Kubernetes Operator is an innovative tool created by Nutanix to streamline the management and operation of the Nutanix NDB (NoSQL database) on Kubernetes clusters.

With the NDB Kubernetes Operator, deploying and managing NDB clusters on Kubernetes has never been easier, as it eliminates the need to manually configure and manage the underlying infrastructure. Built on the Kubernetes operator framework, it offers a declarative way to manage the lifecycle of NDB clusters and other related resources.

One of the key benefits of the operator is that it simplifies the management of NDB clusters by automating common tasks, such as cluster creation, scaling, upgrading, backup, and recovery. It also offers a high degree of flexibility and customization, allowing you to configure various aspects of the cluster, such as storage, networking, and security.

Another advantage of the NDB Kubernetes Operator is its seamless integration with other Kubernetes tools and resources, such as Helm charts, Kubernetes secrets, and Kubernetes ConfigMaps. This integration makes it easy to integrate NDB into your existing Kubernetes-based infrastructure and workflows, providing a hassle-free solution for managing your database clusters.

Overall, the NDB Kubernetes Operator is a powerful and flexible tool for managing NDB clusters on Kubernetes, freeing you up to focus on your application logic rather than infrastructure management. Its automation capabilities and integration with other Kubernetes tools make it a must-have tool for developers and administrators looking to simplify and streamline their database management on Kubernetes.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:file2.png|1300px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
Large amounts of data may be handled by the highly scalable, fault-tolerant, and consistent Nutanix NDB NoSQL database. It is a distributed database created to be installed over several cluster nodes. A portion of the data is stored on each node in the cluster, and the data is replicated across several nodes to guarantee high availability.

Configure your Nutanix cluster: We need to configure your Nutanix cluster to support NDB. This includes setting up the storage and network configurations, configuring the NDB nodes, and defining the replication factor.

Create a table: We need to create a table in NDB to store your data. This includes defining the schema, specifying the replication factor, and configuring any other options you need.

Write your code: We need to write your code to interact with the NDB cluster. This includes inserting and retrieving data, as well as performing more complex operations such as querying, indexing, and data aggregation.

Test your code: We need to test your code to ensure that it works as expected. This includes testing basic operations such as creating and retrieving data, as well as testing more complex operations such as queries and data aggregation.

Monitor your cluster: We need to monitor your NDB cluster to ensure that it is performing as expected. This includes monitoring resource usage, handling errors and exceptions, and optimizing performance.

Optimize your cluster: We need to optimize your NDB cluster over time to ensure that it continues to meet your needs. This includes tuning the configuration, optimizing queries, and scaling the cluster as needed.

Backup and recovery: We need to establish backup and recovery procedures to ensure that your data is protected against data loss or corruption. This includes regularly backing up your data, testing your backups, and establishing procedures for recovering data in case of a disaster.

[[File:SS_3.png|800px]]

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' For any objects that are created, we could instantiate using say method chaining rather than initializing everything with a constructor.

'''Factory:''' Instead of using the regular way (such as the ‘new’ keyword) to instantiate an object, a factory method would be used to do the same.
This pattern can be used if we are creating a superclass for provisioning databases, and subclasses for provisioning different kinds of databases (MongoDB, MySQL, etc). This is because if we want to add another kind of database to our project, and we are creating new databases by conditional checking, our code could get messy. Thus, a factory method could instead create objects in a smarter way for the different database classes (or modules in our case).

'''Facade:''' This pattern could be used for masking the complicated provisioning payloads.

'''Open and Closed principle:''' As per this principle, an interface is open for extension but closed for modification. We could have an interface with provisioning and deprovisioning methods, and interfaces of different databases could extend and reimplement those methods.

'''Adapter''' design pattern could also be added to adapt to different databases

'''DRY(Don’t repeat yourself):''' There are a lot of ways in which DRY principles will be applied in our project:
* Reusing the provisioning function
* Using constants instead of variables
* Extracting all common functionality into reusable modules and functions.

==Demo==

[[File:Image_mongo.png|800px]]

''Here is the demo video for the developed infra:''

https://www.youtube.com/watch?v=lUarCdA8RP0

==Tests==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3
* Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/74
[[File:SS_8.png|800px]]

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

File:F4.png

2023-03-26T02:43:43Z

Kgala2:

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T02:43:28Z

Kgala2: /* Features offered by NDB Service: */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
[[File:f4.png|1000px]]
#Nutanix NDB is a distributed NoSQL database service that is part of the Nutanix platform. Some of the key features of NDB include highly scalable architecture, distributed data storage, support for multiple data models, consistent data, fast data access, automatic sharding, real-time analytics, high availability and fault tolerance, and strong security features.
#With its ability to scale up or down the number of nodes in a cluster, Nutanix NDB provides highly scalable architecture without any downtime. Its distributed architecture ensures high availability and fault tolerance, while its support for multiple data models makes it a versatile database service for a wide range of use cases. Additionally, NDB supports strong consistency and fast data access by caching frequently accessed data in memory, which helps reduce the number of disk reads and improves query performance.
#NDB also provides automatic sharding, which helps ensure that your database can handle large amounts of data. You can use graph queries to analyze relationships between data in real-time, which can help you make more informed decisions. Furthermore, NDB offers high availability and fault tolerance through its distributed architecture and replication features. Lastly, NDB provides strong security features, including role-based access control, data encryption at rest, and network security features.

[[File:f1.jpg|1200px]]

===NDB Kubernetes Operator===

A tool that makes it easier to deploy and administer open-source databases on Kubernetes is the NDB Kubernetes operator. It enables users to declaratively deploy and manage well-known databases like MySQL, PostgreSQL, and MariaDB on Kubernetes. That is one method of using the NDB service. The purpose of Kubernetes is to manage stateless objects. We needed an operator since NDB must be handled in a very specific manner.

The NDB Kubernetes Operator makes it simpler to manage databases in a Kubernetes environment by automating database deployment, scaling, backup, recovery, and monitoring. To automate the deployment and management of databases, it also connects with well-known DevOps tools like Ansible, Jenkins, and Terraform. It enables users to take advantage of Kubernetes' advantages, such as load balancing, service discovery, self-healing, rolling updates, and automatic scaling.

Developers and DevOps teams can concentrate on the high-level features of their applications rather than the low-level intricacies of managing databases using the NDB Kubernetes Operator, which makes application deployment and maintenance more scalable and dependable.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:file2.png|1300px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
Large amounts of data may be handled by the highly scalable, fault-tolerant, and consistent Nutanix NDB NoSQL database. It is a distributed database created to be installed over several cluster nodes. A portion of the data is stored on each node in the cluster, and the data is replicated across several nodes to guarantee high availability.

Configure your Nutanix cluster: We need to configure your Nutanix cluster to support NDB. This includes setting up the storage and network configurations, configuring the NDB nodes, and defining the replication factor.

Create a table: We need to create a table in NDB to store your data. This includes defining the schema, specifying the replication factor, and configuring any other options you need.

Write your code: We need to write your code to interact with the NDB cluster. This includes inserting and retrieving data, as well as performing more complex operations such as querying, indexing, and data aggregation.

Test your code: We need to test your code to ensure that it works as expected. This includes testing basic operations such as creating and retrieving data, as well as testing more complex operations such as queries and data aggregation.

Monitor your cluster: We need to monitor your NDB cluster to ensure that it is performing as expected. This includes monitoring resource usage, handling errors and exceptions, and optimizing performance.

Optimize your cluster: We need to optimize your NDB cluster over time to ensure that it continues to meet your needs. This includes tuning the configuration, optimizing queries, and scaling the cluster as needed.

Backup and recovery: We need to establish backup and recovery procedures to ensure that your data is protected against data loss or corruption. This includes regularly backing up your data, testing your backups, and establishing procedures for recovering data in case of a disaster.

[[File:SS_3.png|800px]]

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' For any objects that are created, we could instantiate using say method chaining rather than initializing everything with a constructor.

'''Factory:''' Instead of using the regular way (such as the ‘new’ keyword) to instantiate an object, a factory method would be used to do the same.
This pattern can be used if we are creating a superclass for provisioning databases, and subclasses for provisioning different kinds of databases (MongoDB, MySQL, etc). This is because if we want to add another kind of database to our project, and we are creating new databases by conditional checking, our code could get messy. Thus, a factory method could instead create objects in a smarter way for the different database classes (or modules in our case).

'''Facade:''' This pattern could be used for masking the complicated provisioning payloads.

'''Open and Closed principle:''' As per this principle, an interface is open for extension but closed for modification. We could have an interface with provisioning and deprovisioning methods, and interfaces of different databases could extend and reimplement those methods.

'''Adapter''' design pattern could also be added to adapt to different databases

'''DRY(Don’t repeat yourself):''' There are a lot of ways in which DRY principles will be applied in our project:
* Reusing the provisioning function
* Using constants instead of variables
* Extracting all common functionality into reusable modules and functions.

==Demo==

[[File:Image_mongo.png|800px]]

''Here is the demo video for the developed infra:''

https://www.youtube.com/watch?v=lUarCdA8RP0

==Tests==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3
* Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/74
[[File:SS_8.png|800px]]

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T02:26:56Z

Kgala2: /* NDB Architecture */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
#Nutanix NDB is a distributed NoSQL database service that is part of the Nutanix platform. Some of the key features of NDB include highly scalable architecture, distributed data storage, support for multiple data models, consistent data, fast data access, automatic sharding, real-time analytics, high availability and fault tolerance, and strong security features.
#With its ability to scale up or down the number of nodes in a cluster, Nutanix NDB provides highly scalable architecture without any downtime. Its distributed architecture ensures high availability and fault tolerance, while its support for multiple data models makes it a versatile database service for a wide range of use cases. Additionally, NDB supports strong consistency and fast data access by caching frequently accessed data in memory, which helps reduce the number of disk reads and improves query performance.
#NDB also provides automatic sharding, which helps ensure that your database can handle large amounts of data. You can use graph queries to analyze relationships between data in real-time, which can help you make more informed decisions. Furthermore, NDB offers high availability and fault tolerance through its distributed architecture and replication features. Lastly, NDB provides strong security features, including role-based access control, data encryption at rest, and network security features.

[[File:f1.jpg|1200px]]

===NDB Kubernetes Operator===

A tool that makes it easier to deploy and administer open-source databases on Kubernetes is the NDB Kubernetes operator. It enables users to declaratively deploy and manage well-known databases like MySQL, PostgreSQL, and MariaDB on Kubernetes. That is one method of using the NDB service. The purpose of Kubernetes is to manage stateless objects. We needed an operator since NDB must be handled in a very specific manner.

The NDB Kubernetes Operator makes it simpler to manage databases in a Kubernetes environment by automating database deployment, scaling, backup, recovery, and monitoring. To automate the deployment and management of databases, it also connects with well-known DevOps tools like Ansible, Jenkins, and Terraform. It enables users to take advantage of Kubernetes' advantages, such as load balancing, service discovery, self-healing, rolling updates, and automatic scaling.

Developers and DevOps teams can concentrate on the high-level features of their applications rather than the low-level intricacies of managing databases using the NDB Kubernetes Operator, which makes application deployment and maintenance more scalable and dependable.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:file2.png|1300px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
Large amounts of data may be handled by the highly scalable, fault-tolerant, and consistent Nutanix NDB NoSQL database. It is a distributed database created to be installed over several cluster nodes. A portion of the data is stored on each node in the cluster, and the data is replicated across several nodes to guarantee high availability.

Configure your Nutanix cluster: We need to configure your Nutanix cluster to support NDB. This includes setting up the storage and network configurations, configuring the NDB nodes, and defining the replication factor.

Create a table: We need to create a table in NDB to store your data. This includes defining the schema, specifying the replication factor, and configuring any other options you need.

Write your code: We need to write your code to interact with the NDB cluster. This includes inserting and retrieving data, as well as performing more complex operations such as querying, indexing, and data aggregation.

Test your code: We need to test your code to ensure that it works as expected. This includes testing basic operations such as creating and retrieving data, as well as testing more complex operations such as queries and data aggregation.

Monitor your cluster: We need to monitor your NDB cluster to ensure that it is performing as expected. This includes monitoring resource usage, handling errors and exceptions, and optimizing performance.

Optimize your cluster: We need to optimize your NDB cluster over time to ensure that it continues to meet your needs. This includes tuning the configuration, optimizing queries, and scaling the cluster as needed.

Backup and recovery: We need to establish backup and recovery procedures to ensure that your data is protected against data loss or corruption. This includes regularly backing up your data, testing your backups, and establishing procedures for recovering data in case of a disaster.

[[File:SS_3.png|800px]]

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' For any objects that are created, we could instantiate using say method chaining rather than initializing everything with a constructor.

'''Factory:''' Instead of using the regular way (such as the ‘new’ keyword) to instantiate an object, a factory method would be used to do the same.
This pattern can be used if we are creating a superclass for provisioning databases, and subclasses for provisioning different kinds of databases (MongoDB, MySQL, etc). This is because if we want to add another kind of database to our project, and we are creating new databases by conditional checking, our code could get messy. Thus, a factory method could instead create objects in a smarter way for the different database classes (or modules in our case).

'''Facade:''' This pattern could be used for masking the complicated provisioning payloads.

'''Open and Closed principle:''' As per this principle, an interface is open for extension but closed for modification. We could have an interface with provisioning and deprovisioning methods, and interfaces of different databases could extend and reimplement those methods.

'''Adapter''' design pattern could also be added to adapt to different databases

'''DRY(Don’t repeat yourself):''' There are a lot of ways in which DRY principles will be applied in our project:
* Reusing the provisioning function
* Using constants instead of variables
* Extracting all common functionality into reusable modules and functions.

==Demo==

[[File:Image_mongo.png|800px]]

''Here is the demo video for the developed infra:''

https://www.youtube.com/watch?v=lUarCdA8RP0

==Tests==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3
* Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/74
[[File:SS_8.png|800px]]

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T02:20:14Z

Kgala2: /* Features offered by NDB Service: */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
#Nutanix NDB is a distributed NoSQL database service that is part of the Nutanix platform. Some of the key features of NDB include highly scalable architecture, distributed data storage, support for multiple data models, consistent data, fast data access, automatic sharding, real-time analytics, high availability and fault tolerance, and strong security features.
#With its ability to scale up or down the number of nodes in a cluster, Nutanix NDB provides highly scalable architecture without any downtime. Its distributed architecture ensures high availability and fault tolerance, while its support for multiple data models makes it a versatile database service for a wide range of use cases. Additionally, NDB supports strong consistency and fast data access by caching frequently accessed data in memory, which helps reduce the number of disk reads and improves query performance.
#NDB also provides automatic sharding, which helps ensure that your database can handle large amounts of data. You can use graph queries to analyze relationships between data in real-time, which can help you make more informed decisions. Furthermore, NDB offers high availability and fault tolerance through its distributed architecture and replication features. Lastly, NDB provides strong security features, including role-based access control, data encryption at rest, and network security features.

[[File:f1.jpg|1200px]]

===NDB Kubernetes Operator===

A tool that makes it easier to deploy and administer open-source databases on Kubernetes is the NDB Kubernetes operator. It enables users to declaratively deploy and manage well-known databases like MySQL, PostgreSQL, and MariaDB on Kubernetes. That is one method of using the NDB service. The purpose of Kubernetes is to manage stateless objects. We needed an operator since NDB must be handled in a very specific manner.

The NDB Kubernetes Operator makes it simpler to manage databases in a Kubernetes environment by automating database deployment, scaling, backup, recovery, and monitoring. To automate the deployment and management of databases, it also connects with well-known DevOps tools like Ansible, Jenkins, and Terraform. It enables users to take advantage of Kubernetes' advantages, such as load balancing, service discovery, self-healing, rolling updates, and automatic scaling.

Developers and DevOps teams can concentrate on the high-level features of their applications rather than the low-level intricacies of managing databases using the NDB Kubernetes Operator, which makes application deployment and maintenance more scalable and dependable.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:file2.png|1000px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
Large amounts of data may be handled by the highly scalable, fault-tolerant, and consistent Nutanix NDB NoSQL database. It is a distributed database created to be installed over several cluster nodes. A portion of the data is stored on each node in the cluster, and the data is replicated across several nodes to guarantee high availability.

Configure your Nutanix cluster: We need to configure your Nutanix cluster to support NDB. This includes setting up the storage and network configurations, configuring the NDB nodes, and defining the replication factor.

Create a table: We need to create a table in NDB to store your data. This includes defining the schema, specifying the replication factor, and configuring any other options you need.

Write your code: We need to write your code to interact with the NDB cluster. This includes inserting and retrieving data, as well as performing more complex operations such as querying, indexing, and data aggregation.

Test your code: We need to test your code to ensure that it works as expected. This includes testing basic operations such as creating and retrieving data, as well as testing more complex operations such as queries and data aggregation.

Monitor your cluster: We need to monitor your NDB cluster to ensure that it is performing as expected. This includes monitoring resource usage, handling errors and exceptions, and optimizing performance.

Optimize your cluster: We need to optimize your NDB cluster over time to ensure that it continues to meet your needs. This includes tuning the configuration, optimizing queries, and scaling the cluster as needed.

Backup and recovery: We need to establish backup and recovery procedures to ensure that your data is protected against data loss or corruption. This includes regularly backing up your data, testing your backups, and establishing procedures for recovering data in case of a disaster.

[[File:SS_3.png|800px]]

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' For any objects that are created, we could instantiate using say method chaining rather than initializing everything with a constructor.

'''Factory:''' Instead of using the regular way (such as the ‘new’ keyword) to instantiate an object, a factory method would be used to do the same.
This pattern can be used if we are creating a superclass for provisioning databases, and subclasses for provisioning different kinds of databases (MongoDB, MySQL, etc). This is because if we want to add another kind of database to our project, and we are creating new databases by conditional checking, our code could get messy. Thus, a factory method could instead create objects in a smarter way for the different database classes (or modules in our case).

'''Facade:''' This pattern could be used for masking the complicated provisioning payloads.

'''Open and Closed principle:''' As per this principle, an interface is open for extension but closed for modification. We could have an interface with provisioning and deprovisioning methods, and interfaces of different databases could extend and reimplement those methods.

'''Adapter''' design pattern could also be added to adapt to different databases

'''DRY(Don’t repeat yourself):''' There are a lot of ways in which DRY principles will be applied in our project:
* Reusing the provisioning function
* Using constants instead of variables
* Extracting all common functionality into reusable modules and functions.

==Demo==

[[File:Image_mongo.png|800px]]

''Here is the demo video for the developed infra:''

https://www.youtube.com/watch?v=lUarCdA8RP0

==Tests==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3
* Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/74
[[File:SS_8.png|800px]]

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

File:File2.png

2023-03-26T02:14:38Z

Kgala2:

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T02:14:04Z

Kgala2: /* NDB Architecture */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
#Nutanix NDB is a distributed NoSQL database service that is part of the Nutanix platform. Some of the key features of NDB include highly scalable architecture, distributed data storage, support for multiple data models, consistent data, fast data access, automatic sharding, real-time analytics, high availability and fault tolerance, and strong security features.
#With its ability to scale up or down the number of nodes in a cluster, Nutanix NDB provides highly scalable architecture without any downtime. Its distributed architecture ensures high availability and fault tolerance, while its support for multiple data models makes it a versatile database service for a wide range of use cases. Additionally, NDB supports strong consistency and fast data access by caching frequently accessed data in memory, which helps reduce the number of disk reads and improves query performance.
#NDB also provides automatic sharding, which helps ensure that your database can handle large amounts of data. You can use graph queries to analyze relationships between data in real-time, which can help you make more informed decisions. Furthermore, NDB offers high availability and fault tolerance through its distributed architecture and replication features. Lastly, NDB provides strong security features, including role-based access control, data encryption at rest, and network security features.

[[File:f1.jpg]]

===NDB Kubernetes Operator===

A tool that makes it easier to deploy and administer open-source databases on Kubernetes is the NDB Kubernetes operator. It enables users to declaratively deploy and manage well-known databases like MySQL, PostgreSQL, and MariaDB on Kubernetes. That is one method of using the NDB service. The purpose of Kubernetes is to manage stateless objects. We needed an operator since NDB must be handled in a very specific manner.

The NDB Kubernetes Operator makes it simpler to manage databases in a Kubernetes environment by automating database deployment, scaling, backup, recovery, and monitoring. To automate the deployment and management of databases, it also connects with well-known DevOps tools like Ansible, Jenkins, and Terraform. It enables users to take advantage of Kubernetes' advantages, such as load balancing, service discovery, self-healing, rolling updates, and automatic scaling.

Developers and DevOps teams can concentrate on the high-level features of their applications rather than the low-level intricacies of managing databases using the NDB Kubernetes Operator, which makes application deployment and maintenance more scalable and dependable.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:file2.png|1000px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
Large amounts of data may be handled by the highly scalable, fault-tolerant, and consistent Nutanix NDB NoSQL database. It is a distributed database created to be installed over several cluster nodes. A portion of the data is stored on each node in the cluster, and the data is replicated across several nodes to guarantee high availability.

Configure your Nutanix cluster: We need to configure your Nutanix cluster to support NDB. This includes setting up the storage and network configurations, configuring the NDB nodes, and defining the replication factor.

Create a table: We need to create a table in NDB to store your data. This includes defining the schema, specifying the replication factor, and configuring any other options you need.

Write your code: We need to write your code to interact with the NDB cluster. This includes inserting and retrieving data, as well as performing more complex operations such as querying, indexing, and data aggregation.

Test your code: We need to test your code to ensure that it works as expected. This includes testing basic operations such as creating and retrieving data, as well as testing more complex operations such as queries and data aggregation.

Monitor your cluster: We need to monitor your NDB cluster to ensure that it is performing as expected. This includes monitoring resource usage, handling errors and exceptions, and optimizing performance.

Optimize your cluster: We need to optimize your NDB cluster over time to ensure that it continues to meet your needs. This includes tuning the configuration, optimizing queries, and scaling the cluster as needed.

Backup and recovery: We need to establish backup and recovery procedures to ensure that your data is protected against data loss or corruption. This includes regularly backing up your data, testing your backups, and establishing procedures for recovering data in case of a disaster.

[[File:SS_3.png|800px]]

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' For any objects that are created, we could instantiate using say method chaining rather than initializing everything with a constructor.

'''Factory:''' Instead of using the regular way (such as the ‘new’ keyword) to instantiate an object, a factory method would be used to do the same.
This pattern can be used if we are creating a superclass for provisioning databases, and subclasses for provisioning different kinds of databases (MongoDB, MySQL, etc). This is because if we want to add another kind of database to our project, and we are creating new databases by conditional checking, our code could get messy. Thus, a factory method could instead create objects in a smarter way for the different database classes (or modules in our case).

'''Facade:''' This pattern could be used for masking the complicated provisioning payloads.

'''Open and Closed principle:''' As per this principle, an interface is open for extension but closed for modification. We could have an interface with provisioning and deprovisioning methods, and interfaces of different databases could extend and reimplement those methods.

'''Adapter''' design pattern could also be added to adapt to different databases

'''DRY(Don’t repeat yourself):''' There are a lot of ways in which DRY principles will be applied in our project:
* Reusing the provisioning function
* Using constants instead of variables
* Extracting all common functionality into reusable modules and functions.

==Demo==

[[File:Image_mongo.png|800px]]

''Here is the demo video for the developed infra:''

https://www.youtube.com/watch?v=lUarCdA8RP0

==Tests==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3
* Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/74
[[File:SS_8.png|800px]]

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T02:11:13Z

Kgala2: /* Design & Workflow */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
#Nutanix NDB is a distributed NoSQL database service that is part of the Nutanix platform. Some of the key features of NDB include highly scalable architecture, distributed data storage, support for multiple data models, consistent data, fast data access, automatic sharding, real-time analytics, high availability and fault tolerance, and strong security features.
#With its ability to scale up or down the number of nodes in a cluster, Nutanix NDB provides highly scalable architecture without any downtime. Its distributed architecture ensures high availability and fault tolerance, while its support for multiple data models makes it a versatile database service for a wide range of use cases. Additionally, NDB supports strong consistency and fast data access by caching frequently accessed data in memory, which helps reduce the number of disk reads and improves query performance.
#NDB also provides automatic sharding, which helps ensure that your database can handle large amounts of data. You can use graph queries to analyze relationships between data in real-time, which can help you make more informed decisions. Furthermore, NDB offers high availability and fault tolerance through its distributed architecture and replication features. Lastly, NDB provides strong security features, including role-based access control, data encryption at rest, and network security features.

[[File:f1.jpg]]

===NDB Kubernetes Operator===

A tool that makes it easier to deploy and administer open-source databases on Kubernetes is the NDB Kubernetes operator. It enables users to declaratively deploy and manage well-known databases like MySQL, PostgreSQL, and MariaDB on Kubernetes. That is one method of using the NDB service. The purpose of Kubernetes is to manage stateless objects. We needed an operator since NDB must be handled in a very specific manner.

The NDB Kubernetes Operator makes it simpler to manage databases in a Kubernetes environment by automating database deployment, scaling, backup, recovery, and monitoring. To automate the deployment and management of databases, it also connects with well-known DevOps tools like Ansible, Jenkins, and Terraform. It enables users to take advantage of Kubernetes' advantages, such as load balancing, service discovery, self-healing, rolling updates, and automatic scaling.

Developers and DevOps teams can concentrate on the high-level features of their applications rather than the low-level intricacies of managing databases using the NDB Kubernetes Operator, which makes application deployment and maintenance more scalable and dependable.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:SS_2.png|1000px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
Large amounts of data may be handled by the highly scalable, fault-tolerant, and consistent Nutanix NDB NoSQL database. It is a distributed database created to be installed over several cluster nodes. A portion of the data is stored on each node in the cluster, and the data is replicated across several nodes to guarantee high availability.

Configure your Nutanix cluster: We need to configure your Nutanix cluster to support NDB. This includes setting up the storage and network configurations, configuring the NDB nodes, and defining the replication factor.

Create a table: We need to create a table in NDB to store your data. This includes defining the schema, specifying the replication factor, and configuring any other options you need.

Write your code: We need to write your code to interact with the NDB cluster. This includes inserting and retrieving data, as well as performing more complex operations such as querying, indexing, and data aggregation.

Test your code: We need to test your code to ensure that it works as expected. This includes testing basic operations such as creating and retrieving data, as well as testing more complex operations such as queries and data aggregation.

Monitor your cluster: We need to monitor your NDB cluster to ensure that it is performing as expected. This includes monitoring resource usage, handling errors and exceptions, and optimizing performance.

Optimize your cluster: We need to optimize your NDB cluster over time to ensure that it continues to meet your needs. This includes tuning the configuration, optimizing queries, and scaling the cluster as needed.

Backup and recovery: We need to establish backup and recovery procedures to ensure that your data is protected against data loss or corruption. This includes regularly backing up your data, testing your backups, and establishing procedures for recovering data in case of a disaster.

[[File:SS_3.png|800px]]

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' For any objects that are created, we could instantiate using say method chaining rather than initializing everything with a constructor.

'''Factory:''' Instead of using the regular way (such as the ‘new’ keyword) to instantiate an object, a factory method would be used to do the same.
This pattern can be used if we are creating a superclass for provisioning databases, and subclasses for provisioning different kinds of databases (MongoDB, MySQL, etc). This is because if we want to add another kind of database to our project, and we are creating new databases by conditional checking, our code could get messy. Thus, a factory method could instead create objects in a smarter way for the different database classes (or modules in our case).

'''Facade:''' This pattern could be used for masking the complicated provisioning payloads.

'''Open and Closed principle:''' As per this principle, an interface is open for extension but closed for modification. We could have an interface with provisioning and deprovisioning methods, and interfaces of different databases could extend and reimplement those methods.

'''Adapter''' design pattern could also be added to adapt to different databases

'''DRY(Don’t repeat yourself):''' There are a lot of ways in which DRY principles will be applied in our project:
* Reusing the provisioning function
* Using constants instead of variables
* Extracting all common functionality into reusable modules and functions.

==Demo==

[[File:Image_mongo.png|800px]]

''Here is the demo video for the developed infra:''

https://www.youtube.com/watch?v=lUarCdA8RP0

==Tests==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3
* Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/74
[[File:SS_8.png|800px]]

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

File:F1.jpg

2023-03-26T01:51:53Z

Kgala2: Kgala2 uploaded a new version of File:F1.jpg

File:F1.jpg

2023-03-26T01:50:50Z

Kgala2:

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T01:50:32Z

Kgala2: /* Features offered by NDB Service: */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
#Nutanix NDB is a distributed NoSQL database service that is part of the Nutanix platform. Some of the key features of NDB include highly scalable architecture, distributed data storage, support for multiple data models, consistent data, fast data access, automatic sharding, real-time analytics, high availability and fault tolerance, and strong security features.
#With its ability to scale up or down the number of nodes in a cluster, Nutanix NDB provides highly scalable architecture without any downtime. Its distributed architecture ensures high availability and fault tolerance, while its support for multiple data models makes it a versatile database service for a wide range of use cases. Additionally, NDB supports strong consistency and fast data access by caching frequently accessed data in memory, which helps reduce the number of disk reads and improves query performance.
#NDB also provides automatic sharding, which helps ensure that your database can handle large amounts of data. You can use graph queries to analyze relationships between data in real-time, which can help you make more informed decisions. Furthermore, NDB offers high availability and fault tolerance through its distributed architecture and replication features. Lastly, NDB provides strong security features, including role-based access control, data encryption at rest, and network security features.

[[File:f1.jpg]]

===NDB Kubernetes Operator===

A tool that makes it easier to deploy and administer open-source databases on Kubernetes is the NDB Kubernetes operator. It enables users to declaratively deploy and manage well-known databases like MySQL, PostgreSQL, and MariaDB on Kubernetes. That is one method of using the NDB service. The purpose of Kubernetes is to manage stateless objects. We needed an operator since NDB must be handled in a very specific manner.

The NDB Kubernetes Operator makes it simpler to manage databases in a Kubernetes environment by automating database deployment, scaling, backup, recovery, and monitoring. To automate the deployment and management of databases, it also connects with well-known DevOps tools like Ansible, Jenkins, and Terraform. It enables users to take advantage of Kubernetes' advantages, such as load balancing, service discovery, self-healing, rolling updates, and automatic scaling.

Developers and DevOps teams can concentrate on the high-level features of their applications rather than the low-level intricacies of managing databases using the NDB Kubernetes Operator, which makes application deployment and maintenance more scalable and dependable.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:SS_2.png|1000px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
Large amounts of data may be handled by the highly scalable, fault-tolerant, and consistent Nutanix NDB NoSQL database. It is a distributed database created to be installed over several cluster nodes. A portion of the data is stored on each node in the cluster, and the data is replicated across several nodes to guarantee high availability.

Configure your Nutanix cluster: You need to configure your Nutanix cluster to support NDB. This includes setting up the storage and network configurations, configuring the NDB nodes, and defining the replication factor.

Create a table: You need to create a table in NDB to store your data. This includes defining the schema, specifying the replication factor, and configuring any other options you need.

Write your code: You need to write your code to interact with the NDB cluster. This includes inserting and retrieving data, as well as performing more complex operations such as querying, indexing, and data aggregation.

Test your code: You need to test your code to ensure that it works as expected. This includes testing basic operations such as creating and retrieving data, as well as testing more complex operations such as queries and data aggregation.

Monitor your cluster: You need to monitor your NDB cluster to ensure that it is performing as expected. This includes monitoring resource usage, handling errors and exceptions, and optimizing performance.

Optimize your cluster: You need to optimize your NDB cluster over time to ensure that it continues to meet your needs. This includes tuning the configuration, optimizing queries, and scaling the cluster as needed.

Backup and recovery: You need to establish backup and recovery procedures to ensure that your data is protected against data loss or corruption. This includes regularly backing up your data, testing your backups, and establishing procedures for recovering data in case of a disaster.

[[File:SS_3.png|800px]]

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' For any objects that are created, we could instantiate using say method chaining rather than initializing everything with a constructor.

'''Factory:''' Instead of using the regular way (such as the ‘new’ keyword) to instantiate an object, a factory method would be used to do the same.
This pattern can be used if we are creating a superclass for provisioning databases, and subclasses for provisioning different kinds of databases (MongoDB, MySQL, etc). This is because if we want to add another kind of database to our project, and we are creating new databases by conditional checking, our code could get messy. Thus, a factory method could instead create objects in a smarter way for the different database classes (or modules in our case).

'''Facade:''' This pattern could be used for masking the complicated provisioning payloads.

'''Open and Closed principle:''' As per this principle, an interface is open for extension but closed for modification. We could have an interface with provisioning and deprovisioning methods, and interfaces of different databases could extend and reimplement those methods.

'''Adapter''' design pattern could also be added to adapt to different databases

'''DRY(Don’t repeat yourself):''' There are a lot of ways in which DRY principles will be applied in our project:
* Reusing the provisioning function
* Using constants instead of variables
* Extracting all common functionality into reusable modules and functions.

==Demo==

[[File:Image_mongo.png|800px]]

''Here is the demo video for the developed infra:''

https://www.youtube.com/watch?v=lUarCdA8RP0

==Tests==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3
* Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/74
[[File:SS_8.png|800px]]

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T01:50:15Z

Kgala2: /* Features offered by NDB Service: */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
#Nutanix NDB is a distributed NoSQL database service that is part of the Nutanix platform. Some of the key features of NDB include highly scalable architecture, distributed data storage, support for multiple data models, consistent data, fast data access, automatic sharding, real-time analytics, high availability and fault tolerance, and strong security features.
#With its ability to scale up or down the number of nodes in a cluster, Nutanix NDB provides highly scalable architecture without any downtime. Its distributed architecture ensures high availability and fault tolerance, while its support for multiple data models makes it a versatile database service for a wide range of use cases. Additionally, NDB supports strong consistency and fast data access by caching frequently accessed data in memory, which helps reduce the number of disk reads and improves query performance.
#NDB also provides automatic sharding, which helps ensure that your database can handle large amounts of data. You can use graph queries to analyze relationships between data in real-time, which can help you make more informed decisions. Furthermore, NDB offers high availability and fault tolerance through its distributed architecture and replication features. Lastly, NDB provides strong security features, including role-based access control, data encryption at rest, and network security features.

[[File:f1.jpg]

===NDB Kubernetes Operator===

A tool that makes it easier to deploy and administer open-source databases on Kubernetes is the NDB Kubernetes operator. It enables users to declaratively deploy and manage well-known databases like MySQL, PostgreSQL, and MariaDB on Kubernetes. That is one method of using the NDB service. The purpose of Kubernetes is to manage stateless objects. We needed an operator since NDB must be handled in a very specific manner.

The NDB Kubernetes Operator makes it simpler to manage databases in a Kubernetes environment by automating database deployment, scaling, backup, recovery, and monitoring. To automate the deployment and management of databases, it also connects with well-known DevOps tools like Ansible, Jenkins, and Terraform. It enables users to take advantage of Kubernetes' advantages, such as load balancing, service discovery, self-healing, rolling updates, and automatic scaling.

Developers and DevOps teams can concentrate on the high-level features of their applications rather than the low-level intricacies of managing databases using the NDB Kubernetes Operator, which makes application deployment and maintenance more scalable and dependable.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:SS_2.png|1000px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
Large amounts of data may be handled by the highly scalable, fault-tolerant, and consistent Nutanix NDB NoSQL database. It is a distributed database created to be installed over several cluster nodes. A portion of the data is stored on each node in the cluster, and the data is replicated across several nodes to guarantee high availability.

Configure your Nutanix cluster: You need to configure your Nutanix cluster to support NDB. This includes setting up the storage and network configurations, configuring the NDB nodes, and defining the replication factor.

Create a table: You need to create a table in NDB to store your data. This includes defining the schema, specifying the replication factor, and configuring any other options you need.

Write your code: You need to write your code to interact with the NDB cluster. This includes inserting and retrieving data, as well as performing more complex operations such as querying, indexing, and data aggregation.

Test your code: You need to test your code to ensure that it works as expected. This includes testing basic operations such as creating and retrieving data, as well as testing more complex operations such as queries and data aggregation.

Monitor your cluster: You need to monitor your NDB cluster to ensure that it is performing as expected. This includes monitoring resource usage, handling errors and exceptions, and optimizing performance.

Optimize your cluster: You need to optimize your NDB cluster over time to ensure that it continues to meet your needs. This includes tuning the configuration, optimizing queries, and scaling the cluster as needed.

Backup and recovery: You need to establish backup and recovery procedures to ensure that your data is protected against data loss or corruption. This includes regularly backing up your data, testing your backups, and establishing procedures for recovering data in case of a disaster.

[[File:SS_3.png|800px]]

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' For any objects that are created, we could instantiate using say method chaining rather than initializing everything with a constructor.

'''Factory:''' Instead of using the regular way (such as the ‘new’ keyword) to instantiate an object, a factory method would be used to do the same.
This pattern can be used if we are creating a superclass for provisioning databases, and subclasses for provisioning different kinds of databases (MongoDB, MySQL, etc). This is because if we want to add another kind of database to our project, and we are creating new databases by conditional checking, our code could get messy. Thus, a factory method could instead create objects in a smarter way for the different database classes (or modules in our case).

'''Facade:''' This pattern could be used for masking the complicated provisioning payloads.

'''Open and Closed principle:''' As per this principle, an interface is open for extension but closed for modification. We could have an interface with provisioning and deprovisioning methods, and interfaces of different databases could extend and reimplement those methods.

'''Adapter''' design pattern could also be added to adapt to different databases

'''DRY(Don’t repeat yourself):''' There are a lot of ways in which DRY principles will be applied in our project:
* Reusing the provisioning function
* Using constants instead of variables
* Extracting all common functionality into reusable modules and functions.

==Demo==

[[File:Image_mongo.png|800px]]

''Here is the demo video for the developed infra:''

https://www.youtube.com/watch?v=lUarCdA8RP0

==Tests==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3
* Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/74
[[File:SS_8.png|800px]]

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

File:SS 1.jpg

2023-03-26T01:48:32Z

Kgala2: Kgala2 uploaded a new version of File:SS 1.jpg

File:SS 1.jpg

2023-03-26T01:46:36Z

Kgala2:

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T01:45:56Z

Kgala2: /* Features offered by NDB Service: */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
#Nutanix NDB is a distributed NoSQL database service that is part of the Nutanix platform. Some of the key features of NDB include highly scalable architecture, distributed data storage, support for multiple data models, consistent data, fast data access, automatic sharding, real-time analytics, high availability and fault tolerance, and strong security features.
#With its ability to scale up or down the number of nodes in a cluster, Nutanix NDB provides highly scalable architecture without any downtime. Its distributed architecture ensures high availability and fault tolerance, while its support for multiple data models makes it a versatile database service for a wide range of use cases. Additionally, NDB supports strong consistency and fast data access by caching frequently accessed data in memory, which helps reduce the number of disk reads and improves query performance.
#NDB also provides automatic sharding, which helps ensure that your database can handle large amounts of data. You can use graph queries to analyze relationships between data in real-time, which can help you make more informed decisions. Furthermore, NDB offers high availability and fault tolerance through its distributed architecture and replication features. Lastly, NDB provides strong security features, including role-based access control, data encryption at rest, and network security features.

[[File:SS_1.jpg|1000px]]

===NDB Kubernetes Operator===

A tool that makes it easier to deploy and administer open-source databases on Kubernetes is the NDB Kubernetes operator. It enables users to declaratively deploy and manage well-known databases like MySQL, PostgreSQL, and MariaDB on Kubernetes. That is one method of using the NDB service. The purpose of Kubernetes is to manage stateless objects. We needed an operator since NDB must be handled in a very specific manner.

The NDB Kubernetes Operator makes it simpler to manage databases in a Kubernetes environment by automating database deployment, scaling, backup, recovery, and monitoring. To automate the deployment and management of databases, it also connects with well-known DevOps tools like Ansible, Jenkins, and Terraform. It enables users to take advantage of Kubernetes' advantages, such as load balancing, service discovery, self-healing, rolling updates, and automatic scaling.

Developers and DevOps teams can concentrate on the high-level features of their applications rather than the low-level intricacies of managing databases using the NDB Kubernetes Operator, which makes application deployment and maintenance more scalable and dependable.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:SS_2.png|1000px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
Large amounts of data may be handled by the highly scalable, fault-tolerant, and consistent Nutanix NDB NoSQL database. It is a distributed database created to be installed over several cluster nodes. A portion of the data is stored on each node in the cluster, and the data is replicated across several nodes to guarantee high availability.

Configure your Nutanix cluster: You need to configure your Nutanix cluster to support NDB. This includes setting up the storage and network configurations, configuring the NDB nodes, and defining the replication factor.

Create a table: You need to create a table in NDB to store your data. This includes defining the schema, specifying the replication factor, and configuring any other options you need.

Write your code: You need to write your code to interact with the NDB cluster. This includes inserting and retrieving data, as well as performing more complex operations such as querying, indexing, and data aggregation.

Test your code: You need to test your code to ensure that it works as expected. This includes testing basic operations such as creating and retrieving data, as well as testing more complex operations such as queries and data aggregation.

Monitor your cluster: You need to monitor your NDB cluster to ensure that it is performing as expected. This includes monitoring resource usage, handling errors and exceptions, and optimizing performance.

Optimize your cluster: You need to optimize your NDB cluster over time to ensure that it continues to meet your needs. This includes tuning the configuration, optimizing queries, and scaling the cluster as needed.

Backup and recovery: You need to establish backup and recovery procedures to ensure that your data is protected against data loss or corruption. This includes regularly backing up your data, testing your backups, and establishing procedures for recovering data in case of a disaster.

[[File:SS_3.png|800px]]

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' For any objects that are created, we could instantiate using say method chaining rather than initializing everything with a constructor.

'''Factory:''' Instead of using the regular way (such as the ‘new’ keyword) to instantiate an object, a factory method would be used to do the same.
This pattern can be used if we are creating a superclass for provisioning databases, and subclasses for provisioning different kinds of databases (MongoDB, MySQL, etc). This is because if we want to add another kind of database to our project, and we are creating new databases by conditional checking, our code could get messy. Thus, a factory method could instead create objects in a smarter way for the different database classes (or modules in our case).

'''Facade:''' This pattern could be used for masking the complicated provisioning payloads.

'''Open and Closed principle:''' As per this principle, an interface is open for extension but closed for modification. We could have an interface with provisioning and deprovisioning methods, and interfaces of different databases could extend and reimplement those methods.

'''Adapter''' design pattern could also be added to adapt to different databases

'''DRY(Don’t repeat yourself):''' There are a lot of ways in which DRY principles will be applied in our project:
* Reusing the provisioning function
* Using constants instead of variables
* Extracting all common functionality into reusable modules and functions.

==Demo==

[[File:Image_mongo.png|800px]]

''Here is the demo video for the developed infra:''

https://www.youtube.com/watch?v=lUarCdA8RP0

==Tests==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3
* Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/74
[[File:SS_8.png|800px]]

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T01:43:22Z

Kgala2: /* Features offered by NDB Service: */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
#Nutanix NDB is a distributed NoSQL database service that is part of the Nutanix platform. Some of the key features of NDB include highly scalable architecture, distributed data storage, support for multiple data models, consistent data, fast data access, automatic sharding, real-time analytics, high availability and fault tolerance, and strong security features.
#With its ability to scale up or down the number of nodes in a cluster, Nutanix NDB provides highly scalable architecture without any downtime. Its distributed architecture ensures high availability and fault tolerance, while its support for multiple data models makes it a versatile database service for a wide range of use cases. Additionally, NDB supports strong consistency and fast data access by caching frequently accessed data in memory, which helps reduce the number of disk reads and improves query performance.
#NDB also provides automatic sharding, which helps ensure that your database can handle large amounts of data. You can use graph queries to analyze relationships between data in real-time, which can help you make more informed decisions. Furthermore, NDB offers high availability and fault tolerance through its distributed architecture and replication features. Lastly, NDB provides strong security features, including role-based access control, data encryption at rest, and network security features.

[[File:SS_1.png|1000px]]

===NDB Kubernetes Operator===

A tool that makes it easier to deploy and administer open-source databases on Kubernetes is the NDB Kubernetes operator. It enables users to declaratively deploy and manage well-known databases like MySQL, PostgreSQL, and MariaDB on Kubernetes. That is one method of using the NDB service. The purpose of Kubernetes is to manage stateless objects. We needed an operator since NDB must be handled in a very specific manner.

The NDB Kubernetes Operator makes it simpler to manage databases in a Kubernetes environment by automating database deployment, scaling, backup, recovery, and monitoring. To automate the deployment and management of databases, it also connects with well-known DevOps tools like Ansible, Jenkins, and Terraform. It enables users to take advantage of Kubernetes' advantages, such as load balancing, service discovery, self-healing, rolling updates, and automatic scaling.

Developers and DevOps teams can concentrate on the high-level features of their applications rather than the low-level intricacies of managing databases using the NDB Kubernetes Operator, which makes application deployment and maintenance more scalable and dependable.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:SS_2.png|1000px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
Large amounts of data may be handled by the highly scalable, fault-tolerant, and consistent Nutanix NDB NoSQL database. It is a distributed database created to be installed over several cluster nodes. A portion of the data is stored on each node in the cluster, and the data is replicated across several nodes to guarantee high availability.

Configure your Nutanix cluster: You need to configure your Nutanix cluster to support NDB. This includes setting up the storage and network configurations, configuring the NDB nodes, and defining the replication factor.

Create a table: You need to create a table in NDB to store your data. This includes defining the schema, specifying the replication factor, and configuring any other options you need.

Write your code: You need to write your code to interact with the NDB cluster. This includes inserting and retrieving data, as well as performing more complex operations such as querying, indexing, and data aggregation.

Test your code: You need to test your code to ensure that it works as expected. This includes testing basic operations such as creating and retrieving data, as well as testing more complex operations such as queries and data aggregation.

Monitor your cluster: You need to monitor your NDB cluster to ensure that it is performing as expected. This includes monitoring resource usage, handling errors and exceptions, and optimizing performance.

Optimize your cluster: You need to optimize your NDB cluster over time to ensure that it continues to meet your needs. This includes tuning the configuration, optimizing queries, and scaling the cluster as needed.

Backup and recovery: You need to establish backup and recovery procedures to ensure that your data is protected against data loss or corruption. This includes regularly backing up your data, testing your backups, and establishing procedures for recovering data in case of a disaster.

[[File:SS_3.png|800px]]

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' For any objects that are created, we could instantiate using say method chaining rather than initializing everything with a constructor.

'''Factory:''' Instead of using the regular way (such as the ‘new’ keyword) to instantiate an object, a factory method would be used to do the same.
This pattern can be used if we are creating a superclass for provisioning databases, and subclasses for provisioning different kinds of databases (MongoDB, MySQL, etc). This is because if we want to add another kind of database to our project, and we are creating new databases by conditional checking, our code could get messy. Thus, a factory method could instead create objects in a smarter way for the different database classes (or modules in our case).

'''Facade:''' This pattern could be used for masking the complicated provisioning payloads.

'''Open and Closed principle:''' As per this principle, an interface is open for extension but closed for modification. We could have an interface with provisioning and deprovisioning methods, and interfaces of different databases could extend and reimplement those methods.

'''Adapter''' design pattern could also be added to adapt to different databases

'''DRY(Don’t repeat yourself):''' There are a lot of ways in which DRY principles will be applied in our project:
* Reusing the provisioning function
* Using constants instead of variables
* Extracting all common functionality into reusable modules and functions.

==Demo==

[[File:Image_mongo.png|800px]]

''Here is the demo video for the developed infra:''

https://www.youtube.com/watch?v=lUarCdA8RP0

==Tests==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3
* Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/74
[[File:SS_8.png|800px]]

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T01:33:12Z

Kgala2: /* Design & Workflow */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
#The full database lifecycle, including database provisioning, scalability, version upgrades, and patch automation, can be managed by users.
#Using a single point of control, the software enables users to manage hundreds to thousands of databases, including Microsoft SQL Server, Oracle, PostgreSQL, MySQL, and MongoDB, on a variety of platforms, including on-premises, colocation facilities, and several public clouds.
#With API connectivity with infrastructure management and development platforms like ServiceNow, users may provide databases for both dev/test and production reasons.
#Users of the program can swiftly apply patches to some or all of their databases to stop the most recent security threats. It also offers role-based access controls to limit access to databases, guaranteeing compliance with legal specifications and industry standards.
#Additionally, it offers tools for data encryption, role-based access control, and audit logging, as well as features for data protection, compliance, and security.
#To automate the deployment and management of databases, it connects with well-known DevOps tools like Ansible, Jenkins, and Terraform.

[[File:SS_1.png|1000px]]

===NDB Kubernetes Operator===

A tool that makes it easier to deploy and administer open-source databases on Kubernetes is the NDB Kubernetes operator. It enables users to declaratively deploy and manage well-known databases like MySQL, PostgreSQL, and MariaDB on Kubernetes. That is one method of using the NDB service. The purpose of Kubernetes is to manage stateless objects. We needed an operator since NDB must be handled in a very specific manner.

The NDB Kubernetes Operator makes it simpler to manage databases in a Kubernetes environment by automating database deployment, scaling, backup, recovery, and monitoring. To automate the deployment and management of databases, it also connects with well-known DevOps tools like Ansible, Jenkins, and Terraform. It enables users to take advantage of Kubernetes' advantages, such as load balancing, service discovery, self-healing, rolling updates, and automatic scaling.

Developers and DevOps teams can concentrate on the high-level features of their applications rather than the low-level intricacies of managing databases using the NDB Kubernetes Operator, which makes application deployment and maintenance more scalable and dependable.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:SS_2.png|1000px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
Large amounts of data may be handled by the highly scalable, fault-tolerant, and consistent Nutanix NDB NoSQL database. It is a distributed database created to be installed over several cluster nodes. A portion of the data is stored on each node in the cluster, and the data is replicated across several nodes to guarantee high availability.

Configure your Nutanix cluster: You need to configure your Nutanix cluster to support NDB. This includes setting up the storage and network configurations, configuring the NDB nodes, and defining the replication factor.

Create a table: You need to create a table in NDB to store your data. This includes defining the schema, specifying the replication factor, and configuring any other options you need.

Write your code: You need to write your code to interact with the NDB cluster. This includes inserting and retrieving data, as well as performing more complex operations such as querying, indexing, and data aggregation.

Test your code: You need to test your code to ensure that it works as expected. This includes testing basic operations such as creating and retrieving data, as well as testing more complex operations such as queries and data aggregation.

Monitor your cluster: You need to monitor your NDB cluster to ensure that it is performing as expected. This includes monitoring resource usage, handling errors and exceptions, and optimizing performance.

Optimize your cluster: You need to optimize your NDB cluster over time to ensure that it continues to meet your needs. This includes tuning the configuration, optimizing queries, and scaling the cluster as needed.

Backup and recovery: You need to establish backup and recovery procedures to ensure that your data is protected against data loss or corruption. This includes regularly backing up your data, testing your backups, and establishing procedures for recovering data in case of a disaster.

[[File:SS_3.png|800px]]

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' For any objects that are created, we could instantiate using say method chaining rather than initializing everything with a constructor.

'''Factory:''' Instead of using the regular way (such as the ‘new’ keyword) to instantiate an object, a factory method would be used to do the same.
This pattern can be used if we are creating a superclass for provisioning databases, and subclasses for provisioning different kinds of databases (MongoDB, MySQL, etc). This is because if we want to add another kind of database to our project, and we are creating new databases by conditional checking, our code could get messy. Thus, a factory method could instead create objects in a smarter way for the different database classes (or modules in our case).

'''Facade:''' This pattern could be used for masking the complicated provisioning payloads.

'''Open and Closed principle:''' As per this principle, an interface is open for extension but closed for modification. We could have an interface with provisioning and deprovisioning methods, and interfaces of different databases could extend and reimplement those methods.

'''Adapter''' design pattern could also be added to adapt to different databases

'''DRY(Don’t repeat yourself):''' There are a lot of ways in which DRY principles will be applied in our project:
* Reusing the provisioning function
* Using constants instead of variables
* Extracting all common functionality into reusable modules and functions.

==Demo==

[[File:Image_mongo.png|800px]]

''Here is the demo video for the developed infra:''

https://www.youtube.com/watch?v=lUarCdA8RP0

==Tests==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3
* Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/74
[[File:SS_8.png|800px]]

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T01:31:34Z

Kgala2: /* Design & Workflow */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
#The full database lifecycle, including database provisioning, scalability, version upgrades, and patch automation, can be managed by users.
#Using a single point of control, the software enables users to manage hundreds to thousands of databases, including Microsoft SQL Server, Oracle, PostgreSQL, MySQL, and MongoDB, on a variety of platforms, including on-premises, colocation facilities, and several public clouds.
#With API connectivity with infrastructure management and development platforms like ServiceNow, users may provide databases for both dev/test and production reasons.
#Users of the program can swiftly apply patches to some or all of their databases to stop the most recent security threats. It also offers role-based access controls to limit access to databases, guaranteeing compliance with legal specifications and industry standards.
#Additionally, it offers tools for data encryption, role-based access control, and audit logging, as well as features for data protection, compliance, and security.
#To automate the deployment and management of databases, it connects with well-known DevOps tools like Ansible, Jenkins, and Terraform.

[[File:SS_1.png|1000px]]

===NDB Kubernetes Operator===

A tool that makes it easier to deploy and administer open-source databases on Kubernetes is the NDB Kubernetes operator. It enables users to declaratively deploy and manage well-known databases like MySQL, PostgreSQL, and MariaDB on Kubernetes. That is one method of using the NDB service. The purpose of Kubernetes is to manage stateless objects. We needed an operator since NDB must be handled in a very specific manner.

The NDB Kubernetes Operator makes it simpler to manage databases in a Kubernetes environment by automating database deployment, scaling, backup, recovery, and monitoring. To automate the deployment and management of databases, it also connects with well-known DevOps tools like Ansible, Jenkins, and Terraform. It enables users to take advantage of Kubernetes' advantages, such as load balancing, service discovery, self-healing, rolling updates, and automatic scaling.

Developers and DevOps teams can concentrate on the high-level features of their applications rather than the low-level intricacies of managing databases using the NDB Kubernetes Operator, which makes application deployment and maintenance more scalable and dependable.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:SS_2.png|1000px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
Large amounts of data may be handled by the highly scalable, fault-tolerant, and consistent Nutanix NDB NoSQL database. It is a distributed database created to be installed over several cluster nodes. A portion of the data is stored on each node in the cluster, and the data is replicated across several nodes to guarantee high availability.

Configure your Nutanix cluster: You need to configure your Nutanix cluster to support NDB. This includes setting up the storage and network configurations, configuring the NDB nodes, and defining the replication factor.

Create a table: You need to create a table in NDB to store your data. This includes defining the schema, specifying the replication factor, and configuring any other options you need.

Write your code: You need to write your code to interact with the NDB cluster. This includes inserting and retrieving data, as well as performing more complex operations such as querying, indexing, and data aggregation.

Test your code: You need to test your code to ensure that it works as expected. This includes testing basic operations such as creating and retrieving data, as well as testing more complex operations such as queries and data aggregation.

Monitor your cluster: You need to monitor your NDB cluster to ensure that it is performing as expected. This includes monitoring resource usage, handling errors and exceptions, and optimizing performance.

Optimize your cluster: You need to optimize your NDB cluster over time to ensure that it continues to meet your needs. This includes tuning the configuration, optimizing queries, and scaling the cluster as needed.

Backup and recovery: You need to establish backup and recovery procedures to ensure that your data is protected against data loss or corruption. This includes regularly backing up your data, testing your backups, and establishing procedures for recovering data in case of a disaster.

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' For any objects that are created, we could instantiate using say method chaining rather than initializing everything with a constructor.

'''Factory:''' Instead of using the regular way (such as the ‘new’ keyword) to instantiate an object, a factory method would be used to do the same.
This pattern can be used if we are creating a superclass for provisioning databases, and subclasses for provisioning different kinds of databases (MongoDB, MySQL, etc). This is because if we want to add another kind of database to our project, and we are creating new databases by conditional checking, our code could get messy. Thus, a factory method could instead create objects in a smarter way for the different database classes (or modules in our case).

'''Facade:''' This pattern could be used for masking the complicated provisioning payloads.

'''Open and Closed principle:''' As per this principle, an interface is open for extension but closed for modification. We could have an interface with provisioning and deprovisioning methods, and interfaces of different databases could extend and reimplement those methods.

'''Adapter''' design pattern could also be added to adapt to different databases

'''DRY(Don’t repeat yourself):''' There are a lot of ways in which DRY principles will be applied in our project:
* Reusing the provisioning function
* Using constants instead of variables
* Extracting all common functionality into reusable modules and functions.

==Demo==

[[File:Image_mongo.png|800px]]

''Here is the demo video for the developed infra:''

https://www.youtube.com/watch?v=lUarCdA8RP0

==Tests==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3
* Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/74
[[File:SS_8.png|800px]]

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T00:46:18Z

Kgala2: /* Features offered by NDB Service: */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
#The full database lifecycle, including database provisioning, scalability, version upgrades, and patch automation, can be managed by users.
#Using a single point of control, the software enables users to manage hundreds to thousands of databases, including Microsoft SQL Server, Oracle, PostgreSQL, MySQL, and MongoDB, on a variety of platforms, including on-premises, colocation facilities, and several public clouds.
#With API connectivity with infrastructure management and development platforms like ServiceNow, users may provide databases for both dev/test and production reasons.
#Users of the program can swiftly apply patches to some or all of their databases to stop the most recent security threats. It also offers role-based access controls to limit access to databases, guaranteeing compliance with legal specifications and industry standards.
#Additionally, it offers tools for data encryption, role-based access control, and audit logging, as well as features for data protection, compliance, and security.
#To automate the deployment and management of databases, it connects with well-known DevOps tools like Ansible, Jenkins, and Terraform.

[[File:SS_1.png|1000px]]

===NDB Kubernetes Operator===

A tool that makes it easier to deploy and administer open-source databases on Kubernetes is the NDB Kubernetes operator. It enables users to declaratively deploy and manage well-known databases like MySQL, PostgreSQL, and MariaDB on Kubernetes. That is one method of using the NDB service. The purpose of Kubernetes is to manage stateless objects. We needed an operator since NDB must be handled in a very specific manner.

The NDB Kubernetes Operator makes it simpler to manage databases in a Kubernetes environment by automating database deployment, scaling, backup, recovery, and monitoring. To automate the deployment and management of databases, it also connects with well-known DevOps tools like Ansible, Jenkins, and Terraform. It enables users to take advantage of Kubernetes' advantages, such as load balancing, service discovery, self-healing, rolling updates, and automatic scaling.

Developers and DevOps teams can concentrate on the high-level features of their applications rather than the low-level intricacies of managing databases using the NDB Kubernetes Operator, which makes application deployment and maintenance more scalable and dependable.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:SS_2.png|1000px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
One of the biggest bottlenecks in connecting a database provisioned by the operator to an application is automatically sharing the database instance's connection details with the application. While sharing the username and password is relatively simple, sharing the database instance host IP with the application pod can be more complicated as the IP is assigned after the database is provisioned. To address this, there are two potential methods: creating a K8s service that maps to the external NDB service endpoint once the database instance is provisioned or creating a configmap with the IP for the database and referencing the configmap in the application pod.

In terms of making the application wait for the database before starting up, there are three options: handling the wait in the application logic (not recommended), failing the application and hence the pod in case of a database connection failure so that Kubernetes can attempt to restart the application pod until it succeeds, or using init-containers to wait on a Kubernetes pod or service.

Out of these options, creating a K8s service that maps to the external NDB service endpoint is the preferred method and is recommended by Google Cloud's Kubernetes Best Practices. This method provides a decoupling between the database instance and the application pod, and an init-container can wait for the service to be ready and start up the application container only after the service and the underlying database instance on NDB are available. Using these two mechanisms together can enable automatic connectivity between the database instance and the application pod(s).

[[File:SS_3.png|800px]]

We will be following the Kubernetes Operator Pattern in the project.

The Kubernetes operator pattern is a way to extend the Kubernetes API by defining custom resources and controllers that manage those resources. Operators automate common deployment, scaling, and management tasks for complex applications, such as databases, message queues, and monitoring systems, that require more than just creating a set of pods.

Here are the key components of the Kubernetes operator pattern:

*'''Custom resource definition (CRD): A CRD defines a new type of Kubernetes resource that can be managed by an operator. The CRD specifies the API schema and validation rules for the custom resource.
'''
[[File:SS_4.png|800px]]

The snippet above shows the configuration of the custom resources we will be provisioning. The server URL and cluster changes every time a new era test drive is created, since a single test drive can only last for a maximum of four hours. This will supposedly provision a MongoDB database on the Nutanix Database Service.

*'''Controller:''' A controller watches for changes to the custom resources and takes actions to ensure the desired state of the resources is maintained. The controller reconciles the actual state of the resources with the desired state by interacting with the Kubernetes API server and other APIs, such as cloud providers or external systems.

*'''Operator business logic:''' The operator business logic implements the custom behavior required to manage the custom resources. This can include creating and deleting Kubernetes resources, interacting with external systems, performing backups and restores, and handling failure scenarios.

*'''Domain-specific language (DSL):''' A DSL provides an abstraction layer that simplifies the implementation of the operator business logic. DSLs can be created using a range of programming languages or tools, such as Ansible or Helm, to express the specific requirements of the target system.

The Kubernetes operator pattern is a powerful way to automate complex tasks and simplify the management of applications running in Kubernetes clusters. By using custom resources and controllers, operators can help you automate tasks like backups, scaling, and failover, freeing up time and resources to focus on higher-level tasks.

'''MongoDB provisioning payload:'''

[[File:SS_5.png|700px]]

The image above shows the MongoDB provisioning payload - which will be sent to the Nutanix Database Service client API for provisioning a Mongo database.

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' For any objects that are created, we could instantiate using say method chaining rather than initializing everything with a constructor.

'''Factory:''' Instead of using the regular way (such as the ‘new’ keyword) to instantiate an object, a factory method would be used to do the same.
This pattern can be used if we are creating a superclass for provisioning databases, and subclasses for provisioning different kinds of databases (MongoDB, MySQL, etc). This is because if we want to add another kind of database to our project, and we are creating new databases by conditional checking, our code could get messy. Thus, a factory method could instead create objects in a smarter way for the different database classes (or modules in our case).

'''Facade:''' This pattern could be used for masking the complicated provisioning payloads.

'''Open and Closed principle:''' As per this principle, an interface is open for extension but closed for modification. We could have an interface with provisioning and deprovisioning methods, and interfaces of different databases could extend and reimplement those methods.

'''Adapter''' design pattern could also be added to adapt to different databases

'''DRY(Don’t repeat yourself):''' There are a lot of ways in which DRY principles will be applied in our project:
* Reusing the provisioning function
* Using constants instead of variables
* Extracting all common functionality into reusable modules and functions.

==Demo==

[[File:Image_mongo.png|800px]]

''Here is the demo video for the developed infra:''

https://www.youtube.com/watch?v=lUarCdA8RP0

==Tests==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3
* Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/74
[[File:SS_8.png|800px]]

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T00:39:38Z

Kgala2: /* NDB Architecture */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
#It enables users to manage the complete database lifecycle, including database provisioning, scaling, version upgrades, and patch automation.
#The product allows users to manage hundreds to thousands of databases, such as Microsoft SQL Server, Oracle, PostgreSQL, MySQL, and MongoDB, across various platforms, including on-premises, colocation facilities, and multiple public clouds, all from a single control point.
#Users can provision databases for both dev/test and production purposes through API integration with infrastructure management and development tools, such as ServiceNow.
#The product enables users to quickly deploy patches across some or all of their databases to prevent the latest security threats. Additionally, it provides role-based access controls to restrict access to databases, ensuring compliance with regulatory requirements and best practices.
#It also provides features for data protection, compliance, and security, including data encryption, role-based access control, and audit logging.
#It integrates with popular DevOps tools like Ansible, Jenkins, and Terraform to automate the deployment and management of databases.

[[File:SS_1.png|1000px]]

===NDB Kubernetes Operator===

A tool that makes it easier to deploy and administer open-source databases on Kubernetes is the NDB Kubernetes operator. It enables users to declaratively deploy and manage well-known databases like MySQL, PostgreSQL, and MariaDB on Kubernetes. That is one method of using the NDB service. The purpose of Kubernetes is to manage stateless objects. We needed an operator since NDB must be handled in a very specific manner.

The NDB Kubernetes Operator makes it simpler to manage databases in a Kubernetes environment by automating database deployment, scaling, backup, recovery, and monitoring. To automate the deployment and management of databases, it also connects with well-known DevOps tools like Ansible, Jenkins, and Terraform. It enables users to take advantage of Kubernetes' advantages, such as load balancing, service discovery, self-healing, rolling updates, and automatic scaling.

Developers and DevOps teams can concentrate on the high-level features of their applications rather than the low-level intricacies of managing databases using the NDB Kubernetes Operator, which makes application deployment and maintenance more scalable and dependable.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:SS_2.png|1000px]]

Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB are just a few of the databases that can have high availability, scalability, and speed thanks to the distributed architecture of the Nutanix Database Service. The hyper-converged infrastructure from Nutanix, which offers a scalable and adaptable platform for handling enterprise workloads, is the foundation around which the architecture is built.

There are various layers in the architecture of the Nutanix Database Service. The Nutanix hyperconverged infrastructure is the basic layer that provides the storage, computing, and networking resources needed to run the databases. The Nutanix Acropolis operating system, which offers the essential virtualization and administration features, sits on top of this layer.

The Nutanix Era layer, which is located above the Nutanix Acropolis layer, offers the Nutanix Database Service the ability to manage databases throughout their existence. The Nutanix Era Manager, a centralized management console that offers a single point of access for controlling the databases across several clouds and data centers, is included in this tier.

The Nutanix Era Orchestrator, which is in charge of automating the provisioning, scaling, patching, and backup of the databases, is another component of the Nutanix Era layer. The Orchestrator offers a declarative approach for specifying the desired state of the databases and is built to work with a variety of databases.

The Nutanix Era Application, a web-based interface that enables database administrators and developers to quickly provision and administer the databases, is the final component of the top layer. A self-service interface for installing databases as well as a number of tools for tracking and troubleshooting database performance are offered by the Era Application.

==Design & Workflow==
One of the biggest bottlenecks in connecting a database provisioned by the operator to an application is automatically sharing the database instance's connection details with the application. While sharing the username and password is relatively simple, sharing the database instance host IP with the application pod can be more complicated as the IP is assigned after the database is provisioned. To address this, there are two potential methods: creating a K8s service that maps to the external NDB service endpoint once the database instance is provisioned or creating a configmap with the IP for the database and referencing the configmap in the application pod.

In terms of making the application wait for the database before starting up, there are three options: handling the wait in the application logic (not recommended), failing the application and hence the pod in case of a database connection failure so that Kubernetes can attempt to restart the application pod until it succeeds, or using init-containers to wait on a Kubernetes pod or service.

Out of these options, creating a K8s service that maps to the external NDB service endpoint is the preferred method and is recommended by Google Cloud's Kubernetes Best Practices. This method provides a decoupling between the database instance and the application pod, and an init-container can wait for the service to be ready and start up the application container only after the service and the underlying database instance on NDB are available. Using these two mechanisms together can enable automatic connectivity between the database instance and the application pod(s).

[[File:SS_3.png|800px]]

We will be following the Kubernetes Operator Pattern in the project.

The Kubernetes operator pattern is a way to extend the Kubernetes API by defining custom resources and controllers that manage those resources. Operators automate common deployment, scaling, and management tasks for complex applications, such as databases, message queues, and monitoring systems, that require more than just creating a set of pods.

Here are the key components of the Kubernetes operator pattern:

*'''Custom resource definition (CRD): A CRD defines a new type of Kubernetes resource that can be managed by an operator. The CRD specifies the API schema and validation rules for the custom resource.
'''
[[File:SS_4.png|800px]]

The snippet above shows the configuration of the custom resources we will be provisioning. The server URL and cluster changes every time a new era test drive is created, since a single test drive can only last for a maximum of four hours. This will supposedly provision a MongoDB database on the Nutanix Database Service.

*'''Controller:''' A controller watches for changes to the custom resources and takes actions to ensure the desired state of the resources is maintained. The controller reconciles the actual state of the resources with the desired state by interacting with the Kubernetes API server and other APIs, such as cloud providers or external systems.

*'''Operator business logic:''' The operator business logic implements the custom behavior required to manage the custom resources. This can include creating and deleting Kubernetes resources, interacting with external systems, performing backups and restores, and handling failure scenarios.

*'''Domain-specific language (DSL):''' A DSL provides an abstraction layer that simplifies the implementation of the operator business logic. DSLs can be created using a range of programming languages or tools, such as Ansible or Helm, to express the specific requirements of the target system.

The Kubernetes operator pattern is a powerful way to automate complex tasks and simplify the management of applications running in Kubernetes clusters. By using custom resources and controllers, operators can help you automate tasks like backups, scaling, and failover, freeing up time and resources to focus on higher-level tasks.

'''MongoDB provisioning payload:'''

[[File:SS_5.png|700px]]

The image above shows the MongoDB provisioning payload - which will be sent to the Nutanix Database Service client API for provisioning a Mongo database.

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' For any objects that are created, we could instantiate using say method chaining rather than initializing everything with a constructor.

'''Factory:''' Instead of using the regular way (such as the ‘new’ keyword) to instantiate an object, a factory method would be used to do the same.
This pattern can be used if we are creating a superclass for provisioning databases, and subclasses for provisioning different kinds of databases (MongoDB, MySQL, etc). This is because if we want to add another kind of database to our project, and we are creating new databases by conditional checking, our code could get messy. Thus, a factory method could instead create objects in a smarter way for the different database classes (or modules in our case).

'''Facade:''' This pattern could be used for masking the complicated provisioning payloads.

'''Open and Closed principle:''' As per this principle, an interface is open for extension but closed for modification. We could have an interface with provisioning and deprovisioning methods, and interfaces of different databases could extend and reimplement those methods.

'''Adapter''' design pattern could also be added to adapt to different databases

'''DRY(Don’t repeat yourself):''' There are a lot of ways in which DRY principles will be applied in our project:
* Reusing the provisioning function
* Using constants instead of variables
* Extracting all common functionality into reusable modules and functions.

==Demo==

[[File:Image_mongo.png|800px]]

''Here is the demo video for the developed infra:''

https://www.youtube.com/watch?v=lUarCdA8RP0

==Tests==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3
* Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/74
[[File:SS_8.png|800px]]

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T00:35:47Z

Kgala2: /* Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified */

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
#It enables users to manage the complete database lifecycle, including database provisioning, scaling, version upgrades, and patch automation.
#The product allows users to manage hundreds to thousands of databases, such as Microsoft SQL Server, Oracle, PostgreSQL, MySQL, and MongoDB, across various platforms, including on-premises, colocation facilities, and multiple public clouds, all from a single control point.
#Users can provision databases for both dev/test and production purposes through API integration with infrastructure management and development tools, such as ServiceNow.
#The product enables users to quickly deploy patches across some or all of their databases to prevent the latest security threats. Additionally, it provides role-based access controls to restrict access to databases, ensuring compliance with regulatory requirements and best practices.
#It also provides features for data protection, compliance, and security, including data encryption, role-based access control, and audit logging.
#It integrates with popular DevOps tools like Ansible, Jenkins, and Terraform to automate the deployment and management of databases.

[[File:SS_1.png|1000px]]

===NDB Kubernetes Operator===

A tool that makes it easier to deploy and administer open-source databases on Kubernetes is the NDB Kubernetes operator. It enables users to declaratively deploy and manage well-known databases like MySQL, PostgreSQL, and MariaDB on Kubernetes. That is one method of using the NDB service. The purpose of Kubernetes is to manage stateless objects. We needed an operator since NDB must be handled in a very specific manner.

The NDB Kubernetes Operator makes it simpler to manage databases in a Kubernetes environment by automating database deployment, scaling, backup, recovery, and monitoring. To automate the deployment and management of databases, it also connects with well-known DevOps tools like Ansible, Jenkins, and Terraform. It enables users to take advantage of Kubernetes' advantages, such as load balancing, service discovery, self-healing, rolling updates, and automatic scaling.

Developers and DevOps teams can concentrate on the high-level features of their applications rather than the low-level intricacies of managing databases using the NDB Kubernetes Operator, which makes application deployment and maintenance more scalable and dependable.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The NDB Kubernetes operator currently uses default compute, network and OS software profiles while provisioning the database. Refactor this module to include optional fields and only if absent, fall back to default.

===NDB Architecture===

[[File:SS_2.png|1000px]]

The Nutanix Database Service architecture is a distributed system that is designed to provide high availability, scalability, and performance for various types of databases, including Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB. The architecture is built on top of Nutanix's hyper converged infrastructure, which provides a scalable and flexible platform for running enterprise workloads.

The Nutanix Database Service architecture consists of several layers. At the bottom layer is the Nutanix hyperconverged infrastructure, which provides storage, compute, and networking resources for running the databases. On top of this layer is the Nutanix Acropolis operating system, which provides the core virtualization and management capabilities.

Above the Nutanix Acropolis layer is the Nutanix Era layer, which provides the database lifecycle management capabilities for the Nutanix Database Service. This layer includes the Nutanix Era Manager, which is a centralized management console that provides a single pane of glass for managing the databases across multiple clouds and data centers.

The Nutanix Era layer also includes the Nutanix Era Orchestrator, which is responsible for automating the provisioning, scaling, patching, and backup of the databases. The Orchestrator is designed to work with various databases and provides a declarative model for defining the desired state of the databases.

Finally, at the top layer is the Nutanix Era Application, which is a web-based interface that allows developers and database administrators to easily provision and manage the databases. The Era Application provides a self-service interface for provisioning databases, as well as a suite of tools for monitoring and troubleshooting database performance.

==Design & Workflow==
One of the biggest bottlenecks in connecting a database provisioned by the operator to an application is automatically sharing the database instance's connection details with the application. While sharing the username and password is relatively simple, sharing the database instance host IP with the application pod can be more complicated as the IP is assigned after the database is provisioned. To address this, there are two potential methods: creating a K8s service that maps to the external NDB service endpoint once the database instance is provisioned or creating a configmap with the IP for the database and referencing the configmap in the application pod.

In terms of making the application wait for the database before starting up, there are three options: handling the wait in the application logic (not recommended), failing the application and hence the pod in case of a database connection failure so that Kubernetes can attempt to restart the application pod until it succeeds, or using init-containers to wait on a Kubernetes pod or service.

Out of these options, creating a K8s service that maps to the external NDB service endpoint is the preferred method and is recommended by Google Cloud's Kubernetes Best Practices. This method provides a decoupling between the database instance and the application pod, and an init-container can wait for the service to be ready and start up the application container only after the service and the underlying database instance on NDB are available. Using these two mechanisms together can enable automatic connectivity between the database instance and the application pod(s).

[[File:SS_3.png|800px]]

We will be following the Kubernetes Operator Pattern in the project.

The Kubernetes operator pattern is a way to extend the Kubernetes API by defining custom resources and controllers that manage those resources. Operators automate common deployment, scaling, and management tasks for complex applications, such as databases, message queues, and monitoring systems, that require more than just creating a set of pods.

Here are the key components of the Kubernetes operator pattern:

*'''Custom resource definition (CRD): A CRD defines a new type of Kubernetes resource that can be managed by an operator. The CRD specifies the API schema and validation rules for the custom resource.
'''
[[File:SS_4.png|800px]]

The snippet above shows the configuration of the custom resources we will be provisioning. The server URL and cluster changes every time a new era test drive is created, since a single test drive can only last for a maximum of four hours. This will supposedly provision a MongoDB database on the Nutanix Database Service.

*'''Controller:''' A controller watches for changes to the custom resources and takes actions to ensure the desired state of the resources is maintained. The controller reconciles the actual state of the resources with the desired state by interacting with the Kubernetes API server and other APIs, such as cloud providers or external systems.

*'''Operator business logic:''' The operator business logic implements the custom behavior required to manage the custom resources. This can include creating and deleting Kubernetes resources, interacting with external systems, performing backups and restores, and handling failure scenarios.

*'''Domain-specific language (DSL):''' A DSL provides an abstraction layer that simplifies the implementation of the operator business logic. DSLs can be created using a range of programming languages or tools, such as Ansible or Helm, to express the specific requirements of the target system.

The Kubernetes operator pattern is a powerful way to automate complex tasks and simplify the management of applications running in Kubernetes clusters. By using custom resources and controllers, operators can help you automate tasks like backups, scaling, and failover, freeing up time and resources to focus on higher-level tasks.

'''MongoDB provisioning payload:'''

[[File:SS_5.png|700px]]

The image above shows the MongoDB provisioning payload - which will be sent to the Nutanix Database Service client API for provisioning a Mongo database.

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' For any objects that are created, we could instantiate using say method chaining rather than initializing everything with a constructor.

'''Factory:''' Instead of using the regular way (such as the ‘new’ keyword) to instantiate an object, a factory method would be used to do the same.
This pattern can be used if we are creating a superclass for provisioning databases, and subclasses for provisioning different kinds of databases (MongoDB, MySQL, etc). This is because if we want to add another kind of database to our project, and we are creating new databases by conditional checking, our code could get messy. Thus, a factory method could instead create objects in a smarter way for the different database classes (or modules in our case).

'''Facade:''' This pattern could be used for masking the complicated provisioning payloads.

'''Open and Closed principle:''' As per this principle, an interface is open for extension but closed for modification. We could have an interface with provisioning and deprovisioning methods, and interfaces of different databases could extend and reimplement those methods.

'''Adapter''' design pattern could also be added to adapt to different databases

'''DRY(Don’t repeat yourself):''' There are a lot of ways in which DRY principles will be applied in our project:
* Reusing the provisioning function
* Using constants instead of variables
* Extracting all common functionality into reusable modules and functions.

==Demo==

[[File:Image_mongo.png|800px]]

''Here is the demo video for the developed infra:''

https://www.youtube.com/watch?v=lUarCdA8RP0

==Tests==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3
* Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/74
[[File:SS_8.png|800px]]

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified

2023-03-26T00:29:51Z

Kgala2: Created page with "==Background== Kubernetes An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. De..."

==Background==
Kubernetes
An open-source container orchestration technology called Kubernetes is used to automatically deploy, scale, and manage containerized applications. Developers can use Kubernetes to distribute and control containerized applications across a dispersed network of servers or PCs. To ensure that the actual state of an application matches the desired state, it uses a declarative model to express the desired state and automatically manages the containerized components. Kubernetes can be operated on public or private cloud infrastructure as well as in-house data centers and offers a wide range of functionality for managing containerized applications, such as autonomous scaling, rolling updates, self-healing, service discovery, and load balancing.

===Nutanix Database Service===

A hybrid multi-cloud database-as-a-service for Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL, among other databases, is called Nutanix Database Service. It allows for the efficient management of hundreds to thousands of databases, the quick creation of new ones, and the automation of time-consuming administration activities like patching and backups. Users can also choose certain operating systems, database versions, and extensions to satisfy application and compliance requirements. Customers from all around the world have optimized their databases across numerous locations and sped up software development using Nutanix Database Service.

===Features offered by NDB Service:===
#It enables users to manage the complete database lifecycle, including database provisioning, scaling, version upgrades, and patch automation.
#The product allows users to manage hundreds to thousands of databases, such as Microsoft SQL Server, Oracle, PostgreSQL, MySQL, and MongoDB, across various platforms, including on-premises, colocation facilities, and multiple public clouds, all from a single control point.
#Users can provision databases for both dev/test and production purposes through API integration with infrastructure management and development tools, such as ServiceNow.
#The product enables users to quickly deploy patches across some or all of their databases to prevent the latest security threats. Additionally, it provides role-based access controls to restrict access to databases, ensuring compliance with regulatory requirements and best practices.
#It also provides features for data protection, compliance, and security, including data encryption, role-based access control, and audit logging.
#It integrates with popular DevOps tools like Ansible, Jenkins, and Terraform to automate the deployment and management of databases.

[[File:SS_1.png|1000px]]

===NDB Kubernetes Operator===

A tool that makes it easier to deploy and administer open-source databases on Kubernetes is the NDB Kubernetes operator. It enables users to declaratively deploy and manage well-known databases like MySQL, PostgreSQL, and MariaDB on Kubernetes. That is one method of using the NDB service. The purpose of Kubernetes is to manage stateless objects. We needed an operator since NDB must be handled in a very specific manner.

The NDB Kubernetes Operator makes it simpler to manage databases in a Kubernetes environment by automating database deployment, scaling, backup, recovery, and monitoring. To automate the deployment and management of databases, it also connects with well-known DevOps tools like Ansible, Jenkins, and Terraform. It enables users to take advantage of Kubernetes' advantages, such as load balancing, service discovery, self-healing, rolling updates, and automatic scaling.

Developers and DevOps teams can concentrate on the high-level features of their applications rather than the low-level intricacies of managing databases using the NDB Kubernetes Operator, which makes application deployment and maintenance more scalable and dependable.

==Existing Architecture and Problem Statement==
===Problem Statement: Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified===

The current version of the NDB Kubernetes operator has limited support for database provisioning, with PostgreSQL being the only database type currently available. As part of this project, we seek to introduce a new database type - MongoDB, and extend the existing interfaces to enable support for NoSQL databases. Subsequently, we will perform end-to-end testing of the provisioning and deprovisioning processes to ensure their smooth functionality.

===NDB Architecture===

[[File:SS_2.png|1000px]]

The Nutanix Database Service architecture is a distributed system that is designed to provide high availability, scalability, and performance for various types of databases, including Microsoft SQL Server, Oracle Database, PostgreSQL, MySQL, and MongoDB. The architecture is built on top of Nutanix's hyper converged infrastructure, which provides a scalable and flexible platform for running enterprise workloads.

The Nutanix Database Service architecture consists of several layers. At the bottom layer is the Nutanix hyperconverged infrastructure, which provides storage, compute, and networking resources for running the databases. On top of this layer is the Nutanix Acropolis operating system, which provides the core virtualization and management capabilities.

Above the Nutanix Acropolis layer is the Nutanix Era layer, which provides the database lifecycle management capabilities for the Nutanix Database Service. This layer includes the Nutanix Era Manager, which is a centralized management console that provides a single pane of glass for managing the databases across multiple clouds and data centers.

The Nutanix Era layer also includes the Nutanix Era Orchestrator, which is responsible for automating the provisioning, scaling, patching, and backup of the databases. The Orchestrator is designed to work with various databases and provides a declarative model for defining the desired state of the databases.

Finally, at the top layer is the Nutanix Era Application, which is a web-based interface that allows developers and database administrators to easily provision and manage the databases. The Era Application provides a self-service interface for provisioning databases, as well as a suite of tools for monitoring and troubleshooting database performance.

==Design & Workflow==
One of the biggest bottlenecks in connecting a database provisioned by the operator to an application is automatically sharing the database instance's connection details with the application. While sharing the username and password is relatively simple, sharing the database instance host IP with the application pod can be more complicated as the IP is assigned after the database is provisioned. To address this, there are two potential methods: creating a K8s service that maps to the external NDB service endpoint once the database instance is provisioned or creating a configmap with the IP for the database and referencing the configmap in the application pod.

In terms of making the application wait for the database before starting up, there are three options: handling the wait in the application logic (not recommended), failing the application and hence the pod in case of a database connection failure so that Kubernetes can attempt to restart the application pod until it succeeds, or using init-containers to wait on a Kubernetes pod or service.

Out of these options, creating a K8s service that maps to the external NDB service endpoint is the preferred method and is recommended by Google Cloud's Kubernetes Best Practices. This method provides a decoupling between the database instance and the application pod, and an init-container can wait for the service to be ready and start up the application container only after the service and the underlying database instance on NDB are available. Using these two mechanisms together can enable automatic connectivity between the database instance and the application pod(s).

[[File:SS_3.png|800px]]

We will be following the Kubernetes Operator Pattern in the project.

The Kubernetes operator pattern is a way to extend the Kubernetes API by defining custom resources and controllers that manage those resources. Operators automate common deployment, scaling, and management tasks for complex applications, such as databases, message queues, and monitoring systems, that require more than just creating a set of pods.

Here are the key components of the Kubernetes operator pattern:

*'''Custom resource definition (CRD): A CRD defines a new type of Kubernetes resource that can be managed by an operator. The CRD specifies the API schema and validation rules for the custom resource.
'''
[[File:SS_4.png|800px]]

The snippet above shows the configuration of the custom resources we will be provisioning. The server URL and cluster changes every time a new era test drive is created, since a single test drive can only last for a maximum of four hours. This will supposedly provision a MongoDB database on the Nutanix Database Service.

*'''Controller:''' A controller watches for changes to the custom resources and takes actions to ensure the desired state of the resources is maintained. The controller reconciles the actual state of the resources with the desired state by interacting with the Kubernetes API server and other APIs, such as cloud providers or external systems.

*'''Operator business logic:''' The operator business logic implements the custom behavior required to manage the custom resources. This can include creating and deleting Kubernetes resources, interacting with external systems, performing backups and restores, and handling failure scenarios.

*'''Domain-specific language (DSL):''' A DSL provides an abstraction layer that simplifies the implementation of the operator business logic. DSLs can be created using a range of programming languages or tools, such as Ansible or Helm, to express the specific requirements of the target system.

The Kubernetes operator pattern is a powerful way to automate complex tasks and simplify the management of applications running in Kubernetes clusters. By using custom resources and controllers, operators can help you automate tasks like backups, scaling, and failover, freeing up time and resources to focus on higher-level tasks.

'''MongoDB provisioning payload:'''

[[File:SS_5.png|700px]]

The image above shows the MongoDB provisioning payload - which will be sent to the Nutanix Database Service client API for provisioning a Mongo database.

==Potential Design Patterns, Principles, and Code Refactoring strategies==

The codebase could be converted into an Object Oriented fashion with classes. Further, here are some of the design patterns we could use:

'''Builder:''' For any objects that are created, we could instantiate using say method chaining rather than initializing everything with a constructor.

'''Factory:''' Instead of using the regular way (such as the ‘new’ keyword) to instantiate an object, a factory method would be used to do the same.
This pattern can be used if we are creating a superclass for provisioning databases, and subclasses for provisioning different kinds of databases (MongoDB, MySQL, etc). This is because if we want to add another kind of database to our project, and we are creating new databases by conditional checking, our code could get messy. Thus, a factory method could instead create objects in a smarter way for the different database classes (or modules in our case).

'''Facade:''' This pattern could be used for masking the complicated provisioning payloads.

'''Open and Closed principle:''' As per this principle, an interface is open for extension but closed for modification. We could have an interface with provisioning and deprovisioning methods, and interfaces of different databases could extend and reimplement those methods.

'''Adapter''' design pattern could also be added to adapt to different databases

'''DRY(Don’t repeat yourself):''' There are a lot of ways in which DRY principles will be applied in our project:
* Reusing the provisioning function
* Using constants instead of variables
* Extracting all common functionality into reusable modules and functions.

==Demo==

[[File:Image_mongo.png|800px]]

''Here is the demo video for the developed infra:''

https://www.youtube.com/watch?v=lUarCdA8RP0

==Tests==
Here is the video for test execution:

https://www.youtube.com/watch?v=O9Z9ypvQJDM

Here’s an example test for '''GeneratingProvisioningRequest'''.

The purpose of the test is to verify that the function generates a valid provisioning request for different database types. The test sets up a test server using the GetServerTestHelper function, creates a ndbclient object, and then iterates over a list of database types (PostgreSQL, MySQL, and MongoDB).

For each database type, the test creates a dbSpec object that defines the desired state of the database instance. It then creates a reqData map that specifies additional provisioning request parameters such as the password and SSH public key. The GenerateProvisioningRequest function is called with the ndbclient, dbSpec, and reqData arguments to generate the actual provisioning request.

The test then checks that the generated request is valid by asserting that the DatabaseType field matches the expected value for the given database type. It also checks that the SoftwareProfileId, SoftwareProfileVersionId, ComputeProfileId, NetworkProfileId, and DbParameterProfileId fields are not empty, and that the TimeMachineInfo.SlaId field is set to NONE_SLA_ID. If any of the assertions fail, the test logs an error message using the t.Errorf or t.Logf functions, indicating which specific assertion failed.

There are also many tests that are handling a lot of niche functionality with regards to generating the provisioning request. For instance, '''TestGenerateProvisioningRequestReturnsErrorIfDBPasswordIsEmpty''' is one. It ensures that the function returns an error when the db password parameter is empty. The test sets up an HTTP server, creates an instance of an ndbclient, and then generates a provisioning request with an empty database password for three different database types: postgres, mysql, and mongodb. If the function GenerateProvisioningRequest returns no error, the test fails with an error message.

[[File:SS_7.png|1000px]]

==Github==
* Repo: https://github.com/karan-47/ndb-operator/tree/feature/ntnx_3
* Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/74
[[File:SS_8.png|800px]]

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru
* Kartiki Bhandakkar

==Contributors==
* Karan Pradeep Gala (kgala2)
* Ashish Joshi (ajoshi24)
* Tilak Satra (trsatra)

==References==
[1] Nutanix. (n.d.). Nutanix Database Service. Retrieved from https://www.nutanix.com/products/database-service

[2] Kubernetes Operator Pattern https://kubernetes.io/docs/concepts/extend-kubernetes/operator

CSC/ECE 517 Spring 2023

2023-03-26T00:19:57Z

Kgala2:

== OSS Projects ==
[[CSC/ECE 517 Spring_2023 -E2306. Refactor user_controller.rb, user.rb and its child classes]]

[[CSC/ECE 517 Spring_2023 - E2320. Reimplement the Question hierarchy]]

[[CSC/ECE 517 Spring_2023 - E2312 + E2313. Reimplement response.rb and responses_controller.rb]]

[[CSC/ECE 517 Spring_2023 - NTNX-1. Support provisioning MongoDb via NDB Kubernetes Operator]]

[[CSC/ECE 517 Spring_2023 - E2316. Reimplement sign_up_sheet_controller.rb]]

[[CSC/ECE 517 Spring_2023 - E2315. Reimplement signed_up_team.rb, sign_up_topic.rb, sign_up_sheet.rb]]

[[CSC/ECE 517 Spring_2023 - E2323. Refactor DueDate functionality from assignment.rb]]

[[CSC/ECE 517 Spring 2023 - E2317: Reimplement participant.rb]]

[[CSC/ECE 517 Spring 2023 - E2314. Reimplement the response map hierarchy]]

[[CSC/ECE 517 Spring 2023- NTNX-4. Extend NDB operator provision postregresql aws]]

[[CSC/ECE 517 Spring 2023 - E2321. Reimplement QuestionnairesController and QuestionsController]]

[[CSC/ECE 517 Spring 2023 - E2305. Grading audit trail]]

[[CSC/ECE 517 Spring 2023 - E2309. Refactor Node model and its subclasses]]

[[CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator]]

[[CSC/ECE 517 Spring 2023 - E2300. Refactor E1858. Github metrics integration]]

[[CSC/ECE 517 Sping 2023 - E2322: Refactor Questionnaire View to display Bookmark Rating]]

[[CSC/ECE 517 Spring 2023 - NTNX-3. Refactor models to keep profiles (software, compute, network, etc) as optional and use default if not specified]]