Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-04-28T20:41:56Z

Pkamath3: /* Tasks Implemented in Project 3 */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented in Project 3==
We set up our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

The ActionArguments was changed as well.

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get databases by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Design Plan for Final Project==

We plan to implement the factory design pattern to further refine and DRY out our code. Our plan is to implement a DatabaseActionArgumentFactory Interface which will introduce a factory design pattern and this will handle the creation of the different types of database action arguments based on the database type.

To achieve this, the DatabaseActionArgumentFactory which has a single method which will create the action arguments. This method will take in a string representing the database type and return action arguments based on the database type passed in as string.

The implementation of this interface will be done by creating different factory structs for each database type, which will implement the method of the interface. These factory structures will be used to create the specific database action argument for each database type.

By using this approach, the code will be more modular and easier to maintain in the future as new database types can be added by simply adding a new factory struct and implementing the CreateActionArguments method for that type. Additionally, the use of an interface will allow for easier testing and mocking of the database action argument creation logic.

Class diagram:
[[File:design_plan.png|center|border]]

==Testing Plan for Final Project==
To test the implementation of the DatabaseActionArgumentFactory, we will be writing unit tests that verify that the factory is correctly creating the appropriate action arguments for each database type.

The steps we will be following for implementation are as follows:

#Create test cases for each of the supported database types (MySQL, Postgres, MongoDB, etc.)
#For each test case, create a mock of the database instance with the appropriate type.
#Call the createActionArguments method on the DatabaseActionArgumentFactory with the mock database instance as input.
#Verify that the output is a list of the correct action arguments for the corresponding database type.
#Repeat the above steps for all the supported database types.

By following these steps, we will be making sure that the DatabaseActionArgumentFactory is correct and that it is creating the correct action arguments for each supported database type.

Additionally, we will also be able to test that the program is able to handle unsupported database types by passing in an unsupported database type and verifying that the output is nil or an empty list.

The system will then be tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

This is similar to what we had done previously to provision a database using the steps given below:

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:Testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied.

[[File:testing_plan_4.png|center|border]]

A successful response is received by the operator.

[[File:testing_plan_5.png|center|border]]

Checked the test drive for status of provisioning.

[[File:testing_plan_6.png|center|border]]

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator

Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/75

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-04-28T20:40:54Z

Pkamath3: /* Design Plan for Final Project */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented in Project 3==
We set up our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get databases by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Design Plan for Final Project==

We plan to implement the factory design pattern to further refine and DRY out our code. Our plan is to implement a DatabaseActionArgumentFactory Interface which will introduce a factory design pattern and this will handle the creation of the different types of database action arguments based on the database type.

To achieve this, the DatabaseActionArgumentFactory which has a single method which will create the action arguments. This method will take in a string representing the database type and return action arguments based on the database type passed in as string.

The implementation of this interface will be done by creating different factory structs for each database type, which will implement the method of the interface. These factory structures will be used to create the specific database action argument for each database type.

By using this approach, the code will be more modular and easier to maintain in the future as new database types can be added by simply adding a new factory struct and implementing the CreateActionArguments method for that type. Additionally, the use of an interface will allow for easier testing and mocking of the database action argument creation logic.

Class diagram:
[[File:design_plan.png|center|border]]

==Testing Plan for Final Project==
To test the implementation of the DatabaseActionArgumentFactory, we will be writing unit tests that verify that the factory is correctly creating the appropriate action arguments for each database type.

The steps we will be following for implementation are as follows:

#Create test cases for each of the supported database types (MySQL, Postgres, MongoDB, etc.)
#For each test case, create a mock of the database instance with the appropriate type.
#Call the createActionArguments method on the DatabaseActionArgumentFactory with the mock database instance as input.
#Verify that the output is a list of the correct action arguments for the corresponding database type.
#Repeat the above steps for all the supported database types.

By following these steps, we will be making sure that the DatabaseActionArgumentFactory is correct and that it is creating the correct action arguments for each supported database type.

Additionally, we will also be able to test that the program is able to handle unsupported database types by passing in an unsupported database type and verifying that the output is nil or an empty list.

The system will then be tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

This is similar to what we had done previously to provision a database using the steps given below:

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:Testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied.

[[File:testing_plan_4.png|center|border]]

A successful response is received by the operator.

[[File:testing_plan_5.png|center|border]]

Checked the test drive for status of provisioning.

[[File:testing_plan_6.png|center|border]]

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator

Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/75

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-04-28T20:40:34Z

Pkamath3: /* Testing Plan for Final Project */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented in Project 3==
We set up our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get databases by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Design Plan for Final Project==

We plan to implement the factory design pattern to further refine and DRY out our code. Our plan is to implement a DatabaseActionArgumentFactory Interface which will introduce a factory design pattern and this will handle the creation of the different types of database action arguments based on the database type.

To achieve this, the DatabaseActionArgumentFactory which has a single method which will create the action arguments. This method will take in a string representing the database type and return action arguments based on the database type passed in as string.

The implementation of this interface will be done by creating different factory structs for each database type, which will implement the method of the interface. These factory structures will be used to create the specific database action argument for each database type.

By using this approach, the code will be more modular and easier to maintain in the future as new database types can be added by simply adding a new factory struct and implementing the CreateActionArguments method for that type. Additionally, the use of an interface will allow for easier testing and mocking of the database action argument creation logic.

[[File:design_plan.png|center|border]]

==Testing Plan for Final Project==
To test the implementation of the DatabaseActionArgumentFactory, we will be writing unit tests that verify that the factory is correctly creating the appropriate action arguments for each database type.

The steps we will be following for implementation are as follows:

#Create test cases for each of the supported database types (MySQL, Postgres, MongoDB, etc.)
#For each test case, create a mock of the database instance with the appropriate type.
#Call the createActionArguments method on the DatabaseActionArgumentFactory with the mock database instance as input.
#Verify that the output is a list of the correct action arguments for the corresponding database type.
#Repeat the above steps for all the supported database types.

By following these steps, we will be making sure that the DatabaseActionArgumentFactory is correct and that it is creating the correct action arguments for each supported database type.

Additionally, we will also be able to test that the program is able to handle unsupported database types by passing in an unsupported database type and verifying that the output is nil or an empty list.

The system will then be tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

This is similar to what we had done previously to provision a database using the steps given below:

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:Testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied.

[[File:testing_plan_4.png|center|border]]

A successful response is received by the operator.

[[File:testing_plan_5.png|center|border]]

Checked the test drive for status of provisioning.

[[File:testing_plan_6.png|center|border]]

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator

Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/75

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-04-08T03:01:01Z

Pkamath3: /* Github */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented in Project 3==
We set up our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get databases by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Design Plan for Final Project==

We plan to implement the factory design pattern to further refine and DRY out our code. Our plan is to implement a DatabaseActionArgumentFactory Interface which will introduce a factory design pattern and this will handle the creation of the different types of database action arguments based on the database type.

To achieve this, the DatabaseActionArgumentFactory which has a single method which will create the action arguments. This method will take in a string representing the database type and return action arguments based on the database type passed in as string.

The implementation of this interface will be done by creating different factory structs for each database type, which will implement the method of the interface. These factory structures will be used to create the specific database action argument for each database type.

By using this approach, the code will be more modular and easier to maintain in the future as new database types can be added by simply adding a new factory struct and implementing the CreateActionArguments method for that type. Additionally, the use of an interface will allow for easier testing and mocking of the database action argument creation logic.

[[File:design_plan.png|center|border]]

==Testing Plan for Final Project==
To test the implementation of the DatabaseActionArgumentFactory, we will be writing unit tests that verify that the factory is correctly creating the appropriate action arguments for each database type.

The steps we will be following for implementation are as follows:

#Create test cases for each of the supported database types (MySQL, Postgres, MongoDB, etc.)
#For each test case, create a mock of the database instance with the appropriate type.
#Call the createActionArguments method on the DatabaseActionArgumentFactory with the mock database instance as input.
#Verify that the output is a list of the correct action arguments for the corresponding database type.
#Repeat the above steps for all the supported database types.

By following these steps, we will be making sure that the DatabaseActionArgumentFactory is correct and that it is creating the correct action arguments for each supported database type.

Additionally, we will also be able to test that the program is able to handle unsupported database types by passing in an unsupported database type and verifying that the output is nil or an empty list.

The system will then be tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

This is similar to what we had done previously to provision a database using the steps given below:

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:Testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied and a successful response is received by the operator.

[[File:testing_plan_4.png|center|border]]

[[File:testing_plan_5.png|center|border]]

[[File:testing_plan_6.png|center|border]]

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator

Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/75

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-04-08T03:00:55Z

Pkamath3: /* Github */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented in Project 3==
We set up our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get databases by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Design Plan for Final Project==

We plan to implement the factory design pattern to further refine and DRY out our code. Our plan is to implement a DatabaseActionArgumentFactory Interface which will introduce a factory design pattern and this will handle the creation of the different types of database action arguments based on the database type.

To achieve this, the DatabaseActionArgumentFactory which has a single method which will create the action arguments. This method will take in a string representing the database type and return action arguments based on the database type passed in as string.

The implementation of this interface will be done by creating different factory structs for each database type, which will implement the method of the interface. These factory structures will be used to create the specific database action argument for each database type.

By using this approach, the code will be more modular and easier to maintain in the future as new database types can be added by simply adding a new factory struct and implementing the CreateActionArguments method for that type. Additionally, the use of an interface will allow for easier testing and mocking of the database action argument creation logic.

[[File:design_plan.png|center|border]]

==Testing Plan for Final Project==
To test the implementation of the DatabaseActionArgumentFactory, we will be writing unit tests that verify that the factory is correctly creating the appropriate action arguments for each database type.

The steps we will be following for implementation are as follows:

#Create test cases for each of the supported database types (MySQL, Postgres, MongoDB, etc.)
#For each test case, create a mock of the database instance with the appropriate type.
#Call the createActionArguments method on the DatabaseActionArgumentFactory with the mock database instance as input.
#Verify that the output is a list of the correct action arguments for the corresponding database type.
#Repeat the above steps for all the supported database types.

By following these steps, we will be making sure that the DatabaseActionArgumentFactory is correct and that it is creating the correct action arguments for each supported database type.

Additionally, we will also be able to test that the program is able to handle unsupported database types by passing in an unsupported database type and verifying that the output is nil or an empty list.

The system will then be tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

This is similar to what we had done previously to provision a database using the steps given below:

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:Testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied and a successful response is received by the operator.

[[File:testing_plan_4.png|center|border]]

[[File:testing_plan_5.png|center|border]]

[[File:testing_plan_6.png|center|border]]

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator
Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/75

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-04-08T02:58:39Z

Pkamath3: /* Testing Plan */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented in Project 3==
We set up our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get databases by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Design Plan for Final Project==

We plan to implement the factory design pattern to further refine and DRY out our code. Our plan is to implement a DatabaseActionArgumentFactory Interface which will introduce a factory design pattern and this will handle the creation of the different types of database action arguments based on the database type.

To achieve this, the DatabaseActionArgumentFactory which has a single method which will create the action arguments. This method will take in a string representing the database type and return action arguments based on the database type passed in as string.

The implementation of this interface will be done by creating different factory structs for each database type, which will implement the method of the interface. These factory structures will be used to create the specific database action argument for each database type.

By using this approach, the code will be more modular and easier to maintain in the future as new database types can be added by simply adding a new factory struct and implementing the CreateActionArguments method for that type. Additionally, the use of an interface will allow for easier testing and mocking of the database action argument creation logic.

[[File:design_plan.png|center|border]]

==Testing Plan for Final Project==
To test the implementation of the DatabaseActionArgumentFactory, we will be writing unit tests that verify that the factory is correctly creating the appropriate action arguments for each database type.

The steps we will be following for implementation are as follows:

#Create test cases for each of the supported database types (MySQL, Postgres, MongoDB, etc.)
#For each test case, create a mock of the database instance with the appropriate type.
#Call the createActionArguments method on the DatabaseActionArgumentFactory with the mock database instance as input.
#Verify that the output is a list of the correct action arguments for the corresponding database type.
#Repeat the above steps for all the supported database types.

By following these steps, we will be making sure that the DatabaseActionArgumentFactory is correct and that it is creating the correct action arguments for each supported database type.

Additionally, we will also be able to test that the program is able to handle unsupported database types by passing in an unsupported database type and verifying that the output is nil or an empty list.

The system will then be tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

This is similar to what we had done previously to provision a database using the steps given below:

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:Testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied and a successful response is received by the operator.

[[File:testing_plan_4.png|center|border]]

[[File:testing_plan_5.png|center|border]]

[[File:testing_plan_6.png|center|border]]

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator
Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pulls

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-04-08T02:58:19Z

Pkamath3: /* Tasks Implemented */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented in Project 3==
We set up our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get databases by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Design Plan for Final Project==

We plan to implement the factory design pattern to further refine and DRY out our code. Our plan is to implement a DatabaseActionArgumentFactory Interface which will introduce a factory design pattern and this will handle the creation of the different types of database action arguments based on the database type.

To achieve this, the DatabaseActionArgumentFactory which has a single method which will create the action arguments. This method will take in a string representing the database type and return action arguments based on the database type passed in as string.

The implementation of this interface will be done by creating different factory structs for each database type, which will implement the method of the interface. These factory structures will be used to create the specific database action argument for each database type.

By using this approach, the code will be more modular and easier to maintain in the future as new database types can be added by simply adding a new factory struct and implementing the CreateActionArguments method for that type. Additionally, the use of an interface will allow for easier testing and mocking of the database action argument creation logic.

[[File:design_plan.png|center|border]]

==Testing Plan==
To test the implementation of the DatabaseActionArgumentFactory, we will be writing unit tests that verify that the factory is correctly creating the appropriate action arguments for each database type.

The steps we will be following for implementation are as follows:

#Create test cases for each of the supported database types (MySQL, Postgres, MongoDB, etc.)
#For each test case, create a mock of the database instance with the appropriate type.
#Call the createActionArguments method on the DatabaseActionArgumentFactory with the mock database instance as input.
#Verify that the output is a list of the correct action arguments for the corresponding database type.
#Repeat the above steps for all the supported database types.

By following these steps, we will be making sure that the DatabaseActionArgumentFactory is correct and that it is creating the correct action arguments for each supported database type.

Additionally, we will also be able to test that the program is able to handle unsupported database types by passing in an unsupported database type and verifying that the output is nil or an empty list.

The system will then be tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

This is similar to what we had done previously to provision a database using the steps given below:

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:Testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied and a successful response is received by the operator.

[[File:testing_plan_4.png|center|border]]

[[File:testing_plan_5.png|center|border]]

[[File:testing_plan_6.png|center|border]]

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator
Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pulls

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-04-08T02:51:23Z

Pkamath3: /* Future Work */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented==
We set up our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get databases by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Design Plan for Final Project==

We plan to implement the factory design pattern to further refine and DRY out our code. Our plan is to implement a DatabaseActionArgumentFactory Interface which will introduce a factory design pattern and this will handle the creation of the different types of database action arguments based on the database type.

To achieve this, the DatabaseActionArgumentFactory which has a single method which will create the action arguments. This method will take in a string representing the database type and return action arguments based on the database type passed in as string.

The implementation of this interface will be done by creating different factory structs for each database type, which will implement the method of the interface. These factory structures will be used to create the specific database action argument for each database type.

By using this approach, the code will be more modular and easier to maintain in the future as new database types can be added by simply adding a new factory struct and implementing the CreateActionArguments method for that type. Additionally, the use of an interface will allow for easier testing and mocking of the database action argument creation logic.

[[File:design_plan.png|center|border]]

==Testing Plan==
To test the implementation of the DatabaseActionArgumentFactory, we will be writing unit tests that verify that the factory is correctly creating the appropriate action arguments for each database type.

The steps we will be following for implementation are as follows:

#Create test cases for each of the supported database types (MySQL, Postgres, MongoDB, etc.)
#For each test case, create a mock of the database instance with the appropriate type.
#Call the createActionArguments method on the DatabaseActionArgumentFactory with the mock database instance as input.
#Verify that the output is a list of the correct action arguments for the corresponding database type.
#Repeat the above steps for all the supported database types.

By following these steps, we will be making sure that the DatabaseActionArgumentFactory is correct and that it is creating the correct action arguments for each supported database type.

Additionally, we will also be able to test that the program is able to handle unsupported database types by passing in an unsupported database type and verifying that the output is nil or an empty list.

The system will then be tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

This is similar to what we had done previously to provision a database using the steps given below:

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:Testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied and a successful response is received by the operator.

[[File:testing_plan_4.png|center|border]]

[[File:testing_plan_5.png|center|border]]

[[File:testing_plan_6.png|center|border]]

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator
Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pulls

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-04-08T02:50:48Z

Pkamath3: /* Testing Plan */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented==
We set up our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get databases by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Design Plan for Final Project==

We plan to implement the factory design pattern to further refine and DRY out our code. Our plan is to implement a DatabaseActionArgumentFactory Interface which will introduce a factory design pattern and this will handle the creation of the different types of database action arguments based on the database type.

To achieve this, the DatabaseActionArgumentFactory which has a single method which will create the action arguments. This method will take in a string representing the database type and return action arguments based on the database type passed in as string.

The implementation of this interface will be done by creating different factory structs for each database type, which will implement the method of the interface. These factory structures will be used to create the specific database action argument for each database type.

By using this approach, the code will be more modular and easier to maintain in the future as new database types can be added by simply adding a new factory struct and implementing the CreateActionArguments method for that type. Additionally, the use of an interface will allow for easier testing and mocking of the database action argument creation logic.

[[File:design_plan.png|center|border]]

==Testing Plan==
To test the implementation of the DatabaseActionArgumentFactory, we will be writing unit tests that verify that the factory is correctly creating the appropriate action arguments for each database type.

The steps we will be following for implementation are as follows:

#Create test cases for each of the supported database types (MySQL, Postgres, MongoDB, etc.)
#For each test case, create a mock of the database instance with the appropriate type.
#Call the createActionArguments method on the DatabaseActionArgumentFactory with the mock database instance as input.
#Verify that the output is a list of the correct action arguments for the corresponding database type.
#Repeat the above steps for all the supported database types.

By following these steps, we will be making sure that the DatabaseActionArgumentFactory is correct and that it is creating the correct action arguments for each supported database type.

Additionally, we will also be able to test that the program is able to handle unsupported database types by passing in an unsupported database type and verifying that the output is nil or an empty list.

The system will then be tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

This is similar to what we had done previously to provision a database using the steps given below:

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:Testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied and a successful response is received by the operator.

[[File:testing_plan_4.png|center|border]]

[[File:testing_plan_5.png|center|border]]

[[File:testing_plan_6.png|center|border]]

==Future Work==
We can currently use the NDB Kubernetes Operator to provision mySQL databases. Deprovisioning the provisioned database would be the most important task for future work. This is essential for freeing up resources and lowering costs. Second, refactoring the code is required to retain the codebase's clean, maintainable, and scalable nature. Finally, it is critical to collaborate and integrate the work of each team to ensure that the project is aligned with the overall company goals and objectives.

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator
Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pulls

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-04-08T02:50:28Z

Pkamath3: /* Testing Plan */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented==
We set up our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get databases by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Design Plan for Final Project==

We plan to implement the factory design pattern to further refine and DRY out our code. Our plan is to implement a DatabaseActionArgumentFactory Interface which will introduce a factory design pattern and this will handle the creation of the different types of database action arguments based on the database type.

To achieve this, the DatabaseActionArgumentFactory which has a single method which will create the action arguments. This method will take in a string representing the database type and return action arguments based on the database type passed in as string.

The implementation of this interface will be done by creating different factory structs for each database type, which will implement the method of the interface. These factory structures will be used to create the specific database action argument for each database type.

By using this approach, the code will be more modular and easier to maintain in the future as new database types can be added by simply adding a new factory struct and implementing the CreateActionArguments method for that type. Additionally, the use of an interface will allow for easier testing and mocking of the database action argument creation logic.

[[File:design_plan.png|center|border]]

==Testing Plan==
To test the implementation of the DatabaseActionArgumentFactory, we will be writing unit tests that verify that the factory is correctly creating the appropriate action arguments for each database type.

The steps we will be following for implementation are as follows:

#Create test cases for each of the supported database types (MySQL, Postgres, MongoDB, etc.)
#For each test case, create a mock of the database instance with the appropriate type.
#Call the createActionArguments method on the DatabaseActionArgumentFactory with the mock database instance as input.
#Verify that the output is a list of the correct action arguments for the corresponding database type.
#Repeat the above steps for all the supported database types.

By following these steps, we will be making sure that the DatabaseActionArgumentFactory is correct and that it is creating the correct action arguments for each supported database type.

Additionally, we will also be able to test that the program is able to handle unsupported database types by passing in an unsupported database type and verifying that the output is nil or an empty list.

The system will then be tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

This is similar to what we had done previously to provision a database using the steps given below:

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:Testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied and a successful response is received by the operator.

[[File:testing_plan_4.png|center|border]]

[[File:testing_plan_5.png|center|border]]

[[File:testing_plan_6.png|center|border]]

==Future Work==
We can currently use the NDB Kubernetes Operator to provision mySQL databases. Deprovisioning the provisioned database would be the most important task for future work. This is essential for freeing up resources and lowering costs. Second, refactoring the code is required to retain the codebase's clean, maintainable, and scalable nature. Finally, it is critical to collaborate and integrate the work of each team to ensure that the project is aligned with the overall company goals and objectives.

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator
Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pulls

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-04-08T02:48:57Z

Pkamath3: /* Design Plan for Final Project */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented==
We set up our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get databases by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Design Plan for Final Project==

We plan to implement the factory design pattern to further refine and DRY out our code. Our plan is to implement a DatabaseActionArgumentFactory Interface which will introduce a factory design pattern and this will handle the creation of the different types of database action arguments based on the database type.

To achieve this, the DatabaseActionArgumentFactory which has a single method which will create the action arguments. This method will take in a string representing the database type and return action arguments based on the database type passed in as string.

The implementation of this interface will be done by creating different factory structs for each database type, which will implement the method of the interface. These factory structures will be used to create the specific database action argument for each database type.

By using this approach, the code will be more modular and easier to maintain in the future as new database types can be added by simply adding a new factory struct and implementing the CreateActionArguments method for that type. Additionally, the use of an interface will allow for easier testing and mocking of the database action argument creation logic.

[[File:design_plan.png|center|border]]

==Testing Plan==
The system was tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:Testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied and a successful response is received by the operator.

[[File:testing_plan_4.png|center|border]]

[[File:testing_plan_5.png|center|border]]

[[File:testing_plan_6.png|center|border]]

==Future Work==
We can currently use the NDB Kubernetes Operator to provision mySQL databases. Deprovisioning the provisioned database would be the most important task for future work. This is essential for freeing up resources and lowering costs. Second, refactoring the code is required to retain the codebase's clean, maintainable, and scalable nature. Finally, it is critical to collaborate and integrate the work of each team to ensure that the project is aligned with the overall company goals and objectives.

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator
Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pulls

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

File:Design plan.png

2023-04-08T02:47:46Z

Pkamath3:

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-04-08T02:44:03Z

Pkamath3: /* D */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented==
We set up our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get databases by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Design Plan for Final Project==

We plan to implement the factory design pattern to further refine and DRY out our code. Our plan is to implement a DatabaseActionArgumentFactory Interface which will introduce a factory design pattern and this will handle the creation of the different types of database action arguments based on the database type.

To achieve this, the DatabaseActionArgumentFactory which has a single method which will create the action arguments. This method will take in a string representing the database type and return action arguments based on the database type passed in as string.

The implementation of this interface will be done by creating different factory structs for each database type, which will implement the method of the interface. These factory structures will be used to create the specific database action argument for each database type.

By using this approach, the code will be more modular and easier to maintain in the future as new database types can be added by simply adding a new factory struct and implementing the CreateActionArguments method for that type. Additionally, the use of an interface will allow for easier testing and mocking of the database action argument creation logic.

==Testing Plan==
The system was tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:Testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied and a successful response is received by the operator.

[[File:testing_plan_4.png|center|border]]

[[File:testing_plan_5.png|center|border]]

[[File:testing_plan_6.png|center|border]]

==Future Work==
We can currently use the NDB Kubernetes Operator to provision mySQL databases. Deprovisioning the provisioned database would be the most important task for future work. This is essential for freeing up resources and lowering costs. Second, refactoring the code is required to retain the codebase's clean, maintainable, and scalable nature. Finally, it is critical to collaborate and integrate the work of each team to ensure that the project is aligned with the overall company goals and objectives.

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator
Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pulls

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-04-08T02:43:21Z

Pkamath3:

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented==
We set up our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get databases by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==D==

==Testing Plan==
The system was tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:Testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied and a successful response is received by the operator.

[[File:testing_plan_4.png|center|border]]

[[File:testing_plan_5.png|center|border]]

[[File:testing_plan_6.png|center|border]]

==Future Work==
We can currently use the NDB Kubernetes Operator to provision mySQL databases. Deprovisioning the provisioned database would be the most important task for future work. This is essential for freeing up resources and lowering costs. Second, refactoring the code is required to retain the codebase's clean, maintainable, and scalable nature. Finally, it is critical to collaborate and integrate the work of each team to ensure that the project is aligned with the overall company goals and objectives.

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator
Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pulls

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-04-08T02:41:53Z

Pkamath3: /* Design Patterns */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented==
We set up our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get databases by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Testing Plan==
The system was tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:Testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied and a successful response is received by the operator.

[[File:testing_plan_4.png|center|border]]

[[File:testing_plan_5.png|center|border]]

[[File:testing_plan_6.png|center|border]]

==Future Work==
We can currently use the NDB Kubernetes Operator to provision mySQL databases. Deprovisioning the provisioned database would be the most important task for future work. This is essential for freeing up resources and lowering costs. Second, refactoring the code is required to retain the codebase's clean, maintainable, and scalable nature. Finally, it is critical to collaborate and integrate the work of each team to ensure that the project is aligned with the overall company goals and objectives.

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator
Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pulls

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

CSC/ECE 517 Spring 2023

2023-03-23T01:14:45Z

Pkamath3:

== OSS Projects ==
[[CSC/ECE 517 Spring_2023 -E2306. Refactor user_controller.rb, user.rb and its child classes]]

[[CSC/ECE 517 Spring_2023 - E2320. Reimplement the Question hierarchy]]

[[CSC/ECE 517 Spring_2023 - E2312 + E2313. Reimplement response.rb and responses_controller.rb]]

[[CSC/ECE 517 Spring_2023 - NTNX-1. Support provisioning MongoDb via NDB Kubernetes Operator]]

[[CSC/ECE 517 Spring_2023 - E2316. Reimplement sign_up_sheet_controller.rb]]

[[CSC/ECE 517 Spring_2023 - E2315. Reimplement signed_up_team.rb, sign_up_topic.rb, sign_up_sheet.rb]]

[[CSC/ECE 517 Spring_2023 - E2323. Refactor DueDate functionality from assignment.rb]]

[[CSC/ECE 517 Spring 2023 - E2314. Reimplement the response map hierarchy]]

[[CSC/ECE 517 Spring 2023- NTNX-4. Extend NDB operator provision postregresql aws]]

[[CSC/ECE 517 Spring 2023 - E2321. Reimplement QuestionnairesController and QuestionsController]]

[[CSC/ECE 517 Spring 2023 - E2305. Grading audit trail]]

[[CSC/ECE 517 Spring 2023 - E2309. Refactor Node model and its subclasses]]

[[CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator]]

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-03-23T00:44:09Z

Pkamath3: /* Future Work */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented==
We setup our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get database by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Testing Plan==
The system was tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:Testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied and a successful response is received by the operator.

[[File:testing_plan_4.png|center|border]]

[[File:testing_plan_5.png|center|border]]

[[File:testing_plan_6.png|center|border]]

==Future Work==
We can currently use the NDB Kubernetes Operator to provision mySQL databases. Deprovisioning the provisioned database would be the most important task for future work. This is essential for freeing up resources and lowering costs. Second, refactoring the code is required to retain the codebase's clean, maintainable, and scalable nature. Finally, it is critical to collaborate and integrate the work of each team to ensure that the project is aligned with the overall company goals and objectives.

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator
Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pulls

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-03-23T00:43:37Z

Pkamath3:

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented==
We setup our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get database by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Testing Plan==
The system was tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:Testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied and a successful response is received by the operator.

[[File:testing_plan_4.png|center|border]]

[[File:testing_plan_5.png|center|border]]

[[File:testing_plan_6.png|center|border]]

==Future Work==
We can currently use the NDB Kubernetes Operator to provision mySQL databases. Deprovisioning the provisioned database would be the most important task for future work. This is essential for freeing up resources and lowering costs. Second, refactoring the remaining code is required to retain the codebase's clean, maintainable, and scalable nature. Finally, it is critical to collaborate and integrate the work of each team to ensure that the project is aligned with the overall company goals and objectives.

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator
Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pulls

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-03-23T00:30:03Z

Pkamath3: /* Testing Plan */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented==
We setup our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get database by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Testing Plan==
The system was tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:Testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied and a successful response is received by the operator.

[[File:testing_plan_4.png|center|border]]

[[File:testing_plan_5.png|center|border]]

[[File:testing_plan_6.png|center|border]]

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator
Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pulls

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-03-23T00:27:34Z

Pkamath3: /* Testing Plan */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented==
We setup our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get database by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Testing Plan==
The system was tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:Testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied and a successful response is received by the operator.

[[File:testing_plan_4.png|center|border]]

[[File:testing_plan_5.png|center|border]]

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator
Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pulls

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

File:Testing plan 3.png

2023-03-23T00:27:09Z

Pkamath3: Pkamath3 uploaded a new version of File:Testing plan 3.png

File:Testing plan 3.png

2023-03-23T00:25:55Z

Pkamath3:

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-03-23T00:25:08Z

Pkamath3: /* Testing Plan */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented==
We setup our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get database by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Testing Plan==
The system was tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied and a successful response is received by the operator.

[[File:testing_plan_4.png|center|border]]

[[File:testing_plan_5.png|center|border]]

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator
Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pulls

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-03-23T00:24:55Z

Pkamath3: /* Testing Plan */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented==
We setup our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get database by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Testing Plan==
The system was tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:tasks_implemented.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied and a successful response is received by the operator.

[[File:testing_plan_4.png|center|border]]

[[File:testing_plan_5.png|center|border]]

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator
Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pulls

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

File:P9 8.png

2023-03-23T00:24:15Z

Pkamath3:

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-03-23T00:14:41Z

Pkamath3: /* Testing Plan */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented==
We setup our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get database by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Testing Plan==
The system was tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:testing_plan_1.png|center|border]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border]]

Step 3 : The secrets were configured

[[File:testing_plan_3.png|center|border]]

Step 4: The configuration for the database was applied and a successful response is received by the operator.

[[File:testing_plan_4.png|center|border]]

[[File:testing_plan_5.png|center|border]]

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator
Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pulls

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-03-23T00:12:51Z

Pkamath3: /* Testing Plan */

==Introduction==
===Nutanix Database Service===
Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database management systems including Microsoft SQL Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. With Nutanix Database Service, you can manage hundreds to thousands of databases with ease. It simplifies tasks like provisioning new databases, automating routine administrative tasks such as backups and patches, and selecting the right operating systems, database versions, and extensions to meet your specific application and compliance requirements.

===Nutanix Database Service Operator===
The Nutanix Database Service (NDB) Operator is an innovative tool that automates and simplifies database administration, provisioning, and life-cycle management on Kubernetes. By utilizing the NDB Operator, developers can directly provision popular database management systems like PostgreSQL, MySQL, and MongoDB from their K8s cluster with ease. This results in significant time savings that could take days to weeks of effort if done manually. The NDB Operator is an open-source tool that can be accessed from your preferred Kubernetes platform, and also allows you to benefit from the full database life-cycle management that NDB provides.

==Problem Statement==
Our team's first task was to get familiar with the operator's codebase which was written in GoLang. We also had to understand basic cluster administration using Kubernetes and make ourselves familiar with the Go Operator SDK which is what is used to develop custom Kubernetes Operators. Our goal was to enhance the functionality of the Nutanix Database Service (NDB) Operator, which at the time could only support the provisioning of PostgreSQL databases. We had to extend the capabilities of the operator to include the provisioning and de-provisioning of MySQL databases. This required modifications to the existing operator to enable the provisioning of MySQL databases, as well as refactoring of the operator's code-base to support this added feature. We also had to make sure that the Operator's code was tested and that the provisioning and de-provisioning steps were tested
end-to-end.

==Tasks Implemented==
We setup our own Kubernetes cluster using Docker Desktop and cloned the ndb-operator repository to make changes to the code.

The manifest file used to generate the provisioning request was set up: This includes the Nutanix Database Service server that is being used as well as parameters such as database instance name, database names, credential secret (which is applied using a different manifest file), size, timezone and type of database that needs to be provisioned.

[[File:tasks_implemented_1.png|center|border|1000px]]

The GenerateProvisioningRequest takes the following arguments: a context, an instance of a struct called NDBClient, an instance of a struct called DatabaseSpec, and a map containing strings as keys and interface{} types as values. The function returns a pointer to an instance of a struct called DatabaseProvisionRequest and an error.

Within the function, a Time Machine Service Level Agreement (SLA) is fetched and as well as OOB profiles from the remote database client. If the function fails to do either of these things, it logs the error and returns.

The function then joins the database instance names together into a single string, and performs type assertions on the password and SSH public key contained within the map argument. If either of these assertions fail (i.e not of type string or empty) errors are thrown.

[[File:tasks_implemented_2.png|center|border|1000px]]

The GetOOBProfiles takes three parameters as input: ctx, ndbclient and dbType and returns a map of profiles and an error. Inside the function, the map of profiles is initialized and GetAllProfiles function is called.

The function then filters out the generic and specific database engine type profiles using the "util.Filter" function. It then again filters these profiles based on the profile type such as COMPUTE, STORAGE, SOFTWARE, NETWORK, and DATABASE_PARAMETER using the same "util.Filter" function.

The function checks if all the required profiles are present or not, and if any of the required profiles is not present, it returns an error. If all the profiles are present, it stores them in the map and returns the map of profiles.

So, basically, this function gets OOB profiles based on the input database engine type and populates a map of profiles.

[[File:tasks_implemented_3.png|center|border|1000px]]

The GetDabaseEngineName and the GetDatabasePortByType functions also returns the database type depending on the definition in the manifest file.

[[File:tasks_implemented_4.png|center|border|1000px]]

The ProvisionDatabase function sends a POST request to the "databases/provision" endpoint of the Network Database service using the ndbClient's Post method. It checks and logs any errors during the POST request. If the POST request is successful and returns a status code of 200 (http.StatusOK), the HTTP response body is read and unmarshaled into the task object.

[[File:tasks_implemented_5.png|center|border|1000px]]

To provision the MySQL database the ActionArguments as well as the port had to be changed.

[[File:tasks_implemented_6.png|center|border|1000px]]

[[File:tasks_implemented_7.png|center|border|1000px]]

==Design Patterns==
One design pattern that can be used to improve this code is the Factory design pattern. The Factory design pattern is used to define an interface for creating an object, but let sub classes decide which class to instantiate. Based on the instance type, the database provision request can be generated and a factory interface can be defined for this. Apart from this the DRY principle has also been followed by making use of profile maps and functions to get database by name in order to have a single method to provision databases and not have different control statements to generate provisioning requests for different database instance types.

==Testing Plan==
The system was tested by running the operator on a local Kubernetes cluster and then making provisioning requests to the Nutanix Database Service test drive cluster that was assigned.

Step 1: First, the manifest for provisioning a mysql database was configured.

[[File:testing_plan_1.png|center|border|1000px]]

Step 2: The NDB operator was then run on a local cluster to make sure that the provisioning request was being completed.

[[File:testing_plan_2.png|center|border|1000px]]

Step 3 : The secrets were configured

[[File:testing_plan_3.png|center|border|1000px]]

Step 4: The configuration for the database was applied and a successful response is received by the operator.

[[File:testing_plan_4.png|center|border|1000px]]

[[File:testing_plan_5.png|center|border|1000px]]

==References==
# NDB Operator Document - https://docs.google.com/document/d/1-VykKyIeky3n4JciIIrNgirk-Cn4pDT1behc9Yl8Nxk/
# NDB Operator Github Repository - https://github.com/nutanix-cloud-native/ndboperator
# Go Documentation - https://go.dev/doc/
# Go Operator SDK Document - https://sdk.operatorframework.io/docs/buildingoperators/golang/tutorial/
# Overview of Kubernetes - https://kubernetes.io/docs/concepts/overview/

==Github==
Repo(Public): https://github.com/qureshi-ali/ndb-operator
Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pulls

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Muhammad Ali Qureshi(mquresh)
* Prasad Vithal Kamath (pkamath3)
* Boscosylvester John Chittilapilly (bchitti)

File:Testing plan 6.png

2023-03-23T00:11:34Z

Pkamath3:

File:Testing plan 5.png

2023-03-23T00:11:24Z

Pkamath3:

File:Testing plan 4.png

2023-03-23T00:11:14Z

Pkamath3:

File:P10 3.png

2023-03-23T00:10:09Z

Pkamath3:

File:Testing plan 2.png

2023-03-23T00:09:57Z

Pkamath3:

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-03-22T23:55:43Z

Pkamath3: /* Tasks Implemented */

File:Tasks implemented 7.png

2023-03-22T23:46:47Z

Pkamath3:

File:Tasks implemented 6.png

2023-03-22T23:46:33Z

Pkamath3:

File:Tasks implemented 5.png

2023-03-22T23:46:20Z

Pkamath3:

File:Tasks implemented 4.png

2023-03-22T23:46:07Z

Pkamath3:

File:Tasks implemented 3.png

2023-03-22T23:45:52Z

Pkamath3:

File:Tasks implemented 2.png

2023-03-22T23:45:25Z

Pkamath3:

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-03-22T23:45:00Z

Pkamath3: /* Tasks Implemented */

File:Tasks implemented 1.png

2023-03-22T23:37:20Z

Pkamath3:

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-03-22T23:24:25Z

Pkamath3:

CSC/ECE 517 Spring 2023 - NTNX-2. Support provisioning mySQL databases via NDB Kubernetes Operator

2023-03-22T23:06:20Z

Pkamath3: Created page with "==Introduction== ===Nutanix Database Service=== Nutanix Database Service is a unique hybrid cloud-based database solution that offers support for several popular database mana..."

CSC/ECE 517 Spring 2023- NTNX-4. Extend NDB operator provision postregresql aws

2023-03-22T23:05:59Z

Pkamath3:

==NTNX-4. Extend Nutanix Database Service Kubernetes operator to provision Postgres Single Instance databases on AWS EC2==

==Note to the reviewer==
A major part of the logic can only be written after we get access to the keys of an EC2 instance that we can connect to the Nutanix Database Service. This differs from other NTNX teams since we need to provision the database on AWS. Hence, we cannot use the Test Nutanix endpoint to create clusterID or Secret's name. We have added a '''Note''' section in the Implementation section to inform the parts which fall under this category. We have also added this information collectively under the '''Future Implementation''' section.

==Problem Statement==
Nutanix Database Service is the only hybrid cloud database-as-a-service for Microsoft SQL
Server, Oracle Database, PostgreSQL, MongoDB, and MySQL. Currently, using the NDB operator, you can only provision a database on the Nutanix Cloud Infrastructure(on-prem).This project aims to extend the Nutanix Database Service Kubernetes operator to provision Postgres Single Instance databases on cloud - specifically on AWS EC2. We aim to do this in a scalable way so that changing the cloud provider to say Azure or GCP would require changes only to the configuration file(CRD Manifest).

==Architecture==
The important components here are:
# '''Kubernetes Cluster'''
# '''Kubernetes Service''' that almost acts like a gateway to the provisioned database - The application pods that want to communicate with the database do so through the K8 service and endpoint that is created as part of the last step of provisioning. The workflow is explained in detail in the next section.
# '''Nutanix Database Service''' - The operator pod running in the K8 cluster communicates with this service to provision, de-provision, obtain status of the database etc.
# '''Cloud platform''' (AWS in our case)

[[File:Ntnx4arch.png|1000px|]]

==Workflow==
* Apply the instance CRD manifest file to create an instance of NDB resource in the K8 cluster

* The operator controller runs its logic to build a provisioning request and sends it to the Nutanix Database Service endpoint pertaining to provision in cloud(AWS in our case)

* The reconcile function loops to take actions depending on the status of provisioning - Empty, Provisioning and Ready

* Once the database is provisioned in AWS and the status is in Ready, we set up a Kubernetes networking service and an endpoint with the same name in the cluster. Any application pod in this cluster will communicate with this endpoint rather than communicating directly with the database.

[[File:Workflowntnx4.png|1000px|]]

==Design==
===Single Responsibility Principle(SRP)===
SRP is adopted here to separate the business logic of the operator from the reconciliation loop.

* The logic to handle '''remoteType''' : cloud is handled separately from the client (ndbClient) through helper methods - This ensures that the client is responsible for only making get/post requests to the specified endpoint without worrying about whether it is for on-prem or cloud.

* The reconciliation loop is responsible for ensuring that the state of the system matches the desired state declared by the operator. It listens to events from the Kubernetes API server and performs the necessary actions to reconcile the system to the desired state. The business logic of the operator, on the other hand, is responsible for implementing the higher-level functionality of the operator, such as managing the deployment of a complex application or managing a database cluster.

* Even within the reconciliation logic, there are clear indications of following SRP. We have a separate file utils/secret.go to implement GetDataFromSecret and GetAllDataFromSecret - logic to get data from the resource denoted by the name/namespace combination.

===Operator Pattern===
The operator pattern is based on the idea of declarative configuration management. Rather than writing imperative code to perform a series of steps to reach a desired state, an operator declares the desired state of the system and uses the Kubernetes API to perform the necessary actions to reconcile the system to that desired state.

We continue to follow this pattern for our implementation. We have a manifest file for a NDB custom resource responsible for provisioning and managing PostgreSQL in AWS and the logic for it is handled in the controller code (controllers/database_controllers.go)

EXAMPLE MANIFEST FILE
...

apiVersion: ndb.nutanix.com/v1alpha1
kind: Database
metadata:
name: db
spec:
ndb:
remoteType: "cloud"
clusterId: "Nutanix Cluster Id"
credentialSecret : ndb-secret-name
server: https://[NDB IP]:8443/era/v1.0
skipCertificateVerification: true
databaseInstance:
databaseInstanceName: "Database Instance Name"
databaseNames:
- database_one
- database_two
- database_three
credentialSecret: db-instance-secret-name
size: 10
timezone: "UTC"
type: postgres
...

The controller logic and related helper code will use the newly created ''''remoteType'''' field to select the NDS endpoint and construct provisionRequest appropriately. We have distinguished the configuration(Manifest file) and the logic(controller code) this way.

==Implementation related to this project==

This section will explain the changes that are made/to-be-made as part of this project.

'''NOTE: MOST OF THESE CHANGES HAVE NOT BEEN MADE NOW SINCE OUR TEAM HAS NOT OBTAINED ACCESS TO CONNECT THE SERVICE WITH EC2 INSTANCE FROM THE NUTANIX TEAM'''
'''We will tell the changes that have not been made and they will be made in the future'''

===Generate Secrets manifest and set name field + credentials in it===
This secrets.yaml manifest file is to create a Secret resource instance that is will have the data regarding the NDB instance name and database credentials.

'''Note: We have created this file, but currently have template values. After getting access, we will fill them accordingly'''

===Modify database_types.go to have allow remote types===
We have added a '''remoteType''' field to the NDB struct along with validation markers
...
// +kubebuilder:validation:Required;Enum=cloud;on-prem
RemoteType string `json:"remoteType"`

===Add helper method to figure out endpoint depending on remote type===
We currently have this logic as part of the ndbClient and should be moved to a separate helper method. The purpose of this function is to determine if the provisioning is for on-prem or cloud.
...
type NDBClient struct {
username string
password string
url string
remoteType string
client *http.Client
}
...
===Modify helper methods to generate request, provision database and setup connectivity===
Below are the functions that are core to the provisioning logic. They are written for provisioning a database in Nutanix cloud infrastructure(on-prem). These functions constitute the primary part of the provisioning workflow.

====GenerateProvisioningRequest(database_reconciler_helpers.go)====
This data structure should be modified/extended to include fields that are required to be part of the JSON request for the NDS endpoint that pertains to provisioning on cloud. Adding fields with appropriate data type is required here to serialize and deserialize requests properly.

====ProvisionDatabase(database_reconciler_helpers.go)====
This method takes care of posting a request to the NDS endpoint to provision database. Should be modified to allow the same on cloud. This method should take care of a helper method logic to determine the correct request data structure and endpoint to be used. We have currently implemented a part of this logic as part of the ndbClient.
'''Note: This can only be implemented after obtaining the access. We are still waiting on it and will implement it as soon as we obtain it.'''

====SetupConnectivity(database_reconciler_helpers.go)====
This function checks and creates a new service (without label selectors) if it does not exists and also sets up the database as the owner for the created service. This also checks and creates an endpoints object for the service if it does not already exists.

'''Note: This is the last part of the serial steps. The previous steps must be completed in order to write this logic.'''

==Test plan==
===Runing tests===
Pull the code, cd into the directory and run:

make test

This runs all the required tests as specified in the Makefile.

===Test Scenarios===
# Testing construction of provision request - All test cases for this that are currently written for on-prem must work for cloud '''remoteType''' too.
# Additional tests for fields that are mandatory for requests constructed for provisioning in cloud
# Verify that the correct API endpoints are being hit based on the remote type specified - On-Prem should hit 0.9 API and cloud should hit 1.0 API
# Test the connectivity between the K8 service and the deployed DB on EC2.

===Actions to be taken before Manual testing===
Manual testing would require you to get access to the Nutanix Database Service

# Obtain access to Nutanix Database Service SASS and access keys for the AWS EC2 instance.

# Add the appropriate credentials to the secrets.yaml file.

# Apply your manifest instance using :
kubectl apply -f <your manifest instance file>

You can verify the status of provisioning in your NDB dashboard.

==Future Implementation==
* Obtain EC2 keys and connect NDS to the EC2 instance
* Construct provision request for provisioning DB in cloud
* Add logic to select proper endpoint and request based on the '''remoteType'''
* Add logic to de-provision the database.

==Github==
Repo(Public): https://github.com/arvindsrinivas1/ndb-operator
Pull Request: https://github.com/nutanix-cloud-native/ndb-operator/pull/73

==Mentors==
* Prof. Edward F. Gehringer
* Krunal Jhaveri
* Manav Rajvanshi
* Krishna Saurabh Vankadaru

==Contributors==
* Arvind Srinivas Subramanian (asubram9)
* Dhanya Sri Dasari (ddasari)
* Zhihao Wang (zwang238@ncsu.edu)

CSC/ECE 517 Spring 2023- NTNX-4. Extend NDB operator provision postregresql aws

2023-03-22T23:05:09Z

Pkamath3: