diff --git a/docs/_static/repositoryserverworkflow.png b/docs/_static/repositoryserverworkflow.png new file mode 100644 index 0000000000..a8481b425b Binary files /dev/null and b/docs/_static/repositoryserverworkflow.png differ diff --git a/docs/architecture.rst b/docs/architecture.rst index 2548ebf17b..b698a70bf5 100644 --- a/docs/architecture.rst +++ b/docs/architecture.rst @@ -6,6 +6,9 @@ Architecture .. contents:: Architecture :local: +Kanister Workflow +================= + The design of Kanister follows the operator pattern. This means Kanister defines its own resources and interacts with those resources through a controller. `This blog post @@ -19,8 +22,6 @@ together: .. image:: ./_static/kanister_workflow.png -Kanister Workflow -================= As seen in the above diagram and described in detail below, all Kanister operations are declarative and require an ActionSet to be @@ -33,6 +34,64 @@ ActionSet. Finally, the original ActionSet will be updated by the controller with status and other metadata generated by the action execution. +Kopia Repository Server Workflow +================================ + +Introducing Kopia +------------------ + +Kopia is a powerful, cross-platform tool for managing encrypted backups +in the cloud. It provides fast and secure backups using compression, +data `de-duplication`, and client-side end-to-end encryption. It supports +a variety of backup storage targets, including object stores, which allows +users to choose the storage provider that best addresses/meets their +requirements. In Kopia, these storage locations are called repositories. +It is a lock-free system that allows concurrent multi-client operations, +including garbage collection. To explore other features of Kopia, +see its `documentation `_ + +Kopia Repository Server +----------------------- + +A Kopia Repository Server allows Kopia clients to proxy access to the backend storage +location through it. At any given time, a repository server can only connect to a single +repository. Due to this limitation, a separate instance of the server will be used +for each repository. + +In Kanister, the server comprises of Kubernetes pod and service. The pod runs +the Kopia repository server process that will be used by Kopia clients to perform +backup and restore operations. Kopia clients only need a username/password and +service name to connect to the server, without requiring to know the backend +storage location. This approach enhances security since only authorized users can +access the Kopia repository server. These authorized users must be added to the +server before starting the server. + +Kopia Repository +---------------- + +The backup storage location is called a "Repository" in Kopia. Only a single +repository can exist at a particular path in the backend storage location. +Users opting to use separate repositories are recommended to use unique path +prefixes for each repository. For example, a repository for a namespace called +monitoring on an S3 storage bucket called test-bucket could be created at the +following location ``s3://test-bucket//repo/``. +Accessing the repository requires the storage location and credential information, +similar to a Kanister Profile CR, and a unique password used by Kopia for +encryption, along with the unique path prefix mentioned above. + + +Repository Server Workflow +-------------------------- +The design of Repository Server consists of a controller and a custom resource +RepositoryServer. The diagram below illustrates the relationship between the repository +server controller and the RepositoryServer custom resource. It shows the integration of +the Kanister controller with the repository server controller and how this integration +can be leveraged to use Kopia as a data mover tool. + + .. image:: ./_static/repositoryserverworkflow.png + +To know more about the design of the Kopia repository server controller, refer to +its `design documentation `_. Custom Resources ================ @@ -46,7 +105,8 @@ objects can be managed entirely through kubectl. Kanister uses Kubernetes' code generation tools to create go client libraries for its CRs. The schemas of the Kanister CRDs can be found in `types.go -`_ +`_ and +`repositoryserver_types.go `_ Blueprints ---------- @@ -363,9 +423,391 @@ As a reference, below is an example of a Profile and the corresponding secret. example_key_id: example_secret_access_key: +.. _repository_servers: + +RepositoryServers +----------------- + +RepositoryServer CR is required by the repository server controller to start +a Kopia repository server. The CR has a list of parameters to configure +the Kopia repository server. + +.. note:: + Secrets referenced in the CR should be created in the format referenced + in the :ref:`Repository Server Secrets` section. + + +The definition of ``Repository Server`` is: + +.. code-block:: go + :linenos: + + // RepositoryServer manages the lifecycle of Kopia Repository Server within a Pod + type RepositoryServer struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + Spec RepositoryServerSpec `json:"spec"` + Status RepositoryServerStatus `json:"status"` + } + + +Repository Server ``Spec`` field is defined as follows: + +.. code-block:: go + :linenos: + + type RepositoryServerSpec struct { + Storage Storage `json:"storage"` + Repository Repository `json:"repository"` + Server Server `json:"server"` + } + + type Storage struct { + SecretRef corev1.SecretReference `json:"secretRef"` + CredentialSecretRef corev1.SecretReference `json:"credentialSecretRef"` + } + +- The ``Storage`` field in the ``RepositoryServerSpec`` contains the location + details where the Kopia repository is created. + + - ``SecretRef`` and ``CredentialSecretRef`` are the references to location + secrets. + +- The ``Repository`` field in the CR ``spec`` contains details for connecting to + the Kopia repository created in the location storage mentioned above. + +.. code-block:: go + :linenos: + + type Repository struct { + RootPath string `json:"rootPath"` + Username string `json:"username"` + Hostname string `json:"hostname"` + PasswordSecretRef corev1.SecretReference `json:"passwordSecretRef"` + CacheSizeSettings CacheSizeSettings `json:"cacheSizeSettings,omitempty"` + } + + type CacheSizeSettings struct { + Metadata string `json:"metadata"` + Content string `json:"content"` + } + + +- ``RootPath`` is the path for the Kopia repository. It is the sub-path within + the path prefix specified in the storage location. +- ``Username`` is an optional field used to override the default username while + connecting to the Kopia repository. +- ``Hostname`` is an optional field used to override the default hostname while + connecting to the Kopia repository. + +Kopia identifies users by ``username@hostname`` and uses the values specified when +establishing a connection to the repository to identify backups created in the session. + +- ``PasswordSecretRef`` is the reference to the secret containing the password to + connect to the Kopia repository. + +- ``CacheSizeSettings`` is an optional field used to specify the size of the different + caches for the Kopia repository. If not specified, default cache settings are used + by repository server controller. + +To know more about the Kopia caches, refer to the `Kopia caching documentation `_. + +- ``Server`` field in the CR spec has references to all the secrets + required to start the Kopia repository server. + +.. code-block:: go + :linenos: + + type Server struct { + UserAccess UserAccess `json:"userAccess"` + AdminSecretRef corev1.SecretReference `json:"adminSecretRef"` + TLSSecretRef corev1.SecretReference `json:"tlsSecretRef"` + } + + type UserAccess struct { + UserAccessSecretRef corev1.SecretReference `json:"userAccessSecretRef"` + Username string `json:"username"` + } + +- ``AdminSecretRef`` is a secret reference containing admin credentials + required to start the Kopia repository server. + +- ``TLSSecretRef`` is a TLS secret reference for Kopia client and server communication. + +- The ``UserAccess`` field contains a username and password secret reference required + for creating Kopia repository server users. + +- The ``Status`` field in the ``RepositoryServer`` CR is used by repository server + controller to propagate the server's status to the client. It is defined as: + +.. code-block:: go + :linenos: + + type RepositoryServerStatus struct { + Conditions []metav1.Condition `json:"conditions,omitempty"` + ServerInfo ServerInfo `json:"serverInfo,omitempty"` + Progress RepositoryServerProgress `json:"progress"` + } + + type ServerInfo struct { + PodName string `json:"podName,omitempty"` + ServiceName string `json:"serviceName,omitempty"` + } + +- ``Progress`` is populated by the controller with three values: + + - ``Ready`` represents the ready state of the repository server and + the pod, which runs the proxy server. + + - ``Failed`` represents that the controller got an error while + starting the repository server. + + - ``Pending`` represents that the repository server has not yet completely started. + +- ``ServerInfo`` is populated by the repository server controller with + the server details that the client requires to connect to the server. + + - ``PodName`` is the name of the pod created by the controller for the + Kopia repository server. + + - ``ServiceName`` is the name of the Kubernetes service created by the controller, + which contains the connection details for the repository server. + + +As a reference, below is an example of a Repository Server: + +.. code-block:: yaml + :linenos: + + apiVersion: cr.kanister.io/v1alpha1 + kind: RepositoryServer + metadata: + name: kopia-repo-server + namespace: + spec: + storage: + secretRef: + name: + namespace: + credentialSecretRef: + name: + namespace: + repository: + rootPath: + passwordSecretRef: + name: + namespace: + username: + hostname: + server: + adminSecretRef: + name: + namespace: + tlsSecretRef: + name: + namespace: + userAccess: + userAccessSecretRef: + name: + namespace: + username: + +.. _repository_Server_secrets: + +Repository Server Secrets +========================= + +The repository server controller needs the following secrets to be created for starting +the Kopia repository server successfully. These secrets are referenced in the +``RepositoryServer`` CR, as described in the :ref:`RepositoryServer`. + +Location Storage Secret +----------------------- + +This secret stores the sensitive details of the location where the Kopia +repository is created. It is referenced by the ``spec.storage.secretRef`` +field in the repository server CR. + +The ``data.type`` field can have following values ``s3``, ``gcs``, +``azure``, and ``file-store``. + +.. code-block:: yaml + :linenos: + + apiVersion: v1 + kind: Secret + metadata: + name: location + namespace: + type: secrets.kanister.io/storage-location + data: + # required: specify the type of the store + # supported values are s3, gcs, azure, and file-store + type: + # required + bucket: + # optional: specified in case of S3-compatible stores + endpoint: + # optional: used as a sub path in the bucket for all backups + path: + # required, if supported by the provider + region: + # required: if type is `file-store` + # optional, otherwise + claimName: + +Location Credentials Secret +--------------------------- + +The following secret should be used for Azure, AWS, and GCS storage credentials. +This secret is referenced by the ``spec.storage.credentialSecretRef`` in the +repository server CR: + +- ``AWS S3`` + +.. code-block:: yaml + :linenos: + + apiVersion: v1 + kind: Secret + metadata: + name: s3-loc-creds + namespace: + type: secrets.kanister.io/aws + data: + # required: base64 encoded value for key with proper permissions for the bucket + access-key: + # required: base64 encoded value for the secret corresponding to the key above + secret-acccess-key: + # optional: base64 encoded value for AWS IAM role + role: + +- ``Azure`` + +.. code-block:: yaml + :linenos: + + apiVersion: v1 + kind: Secret + metadata: + name: s3-loc-creds + namespace: + type: secrets.kanister.io/aws + data: + # required: base64 encoded value for account with proper permissions for the bucket + azure_storage_account_id: + # required: base64 encoded value for the key corresponding to the account above + azure_storage_key: + # optional: base64 encoded value for the storage enevironment. + # Acceptable values are AzureCloud, AzureChinaCloud, AzureUSGovernment, AzureGermanCloud + azure_storage_environment: + +- ``GCS`` + + .. code-block:: yaml + :linenos: + + apiVersion: v1 + kind: Secret + metadata: + name: gcs-loc-creds + namespace: + type: secrets.kanister.io/gcp + data: + # required: base64 encoded value for project with proper permissions for the bucket + project-id: + # required: base64 encoded value for the SA with proper permissions for the bucket. + # This value is base64 encoding of the service account json file when + # creating a new service account + service-account.json: + + +Repository Password Secret +-------------------------- +This is the secret password used by the controller to connect to the Kopia repository. +It is referenced by the ``spec.repository.passwordSecretRef`` in the repository server CR. + +.. code-block:: yaml + :linenos: + + apiVersion: v1 + kind: Secret + metadata: + name: repository-password + namespace: + type: secrets.kanister.io/kopia-repository/password + data: + repo-password: + +Repository Server Admin User Secret +----------------------------------- +This secret is used for storing admin credentials that are used by the controller +to start the Kopia repository server. It is referenced by the +``spec.server.accessSecretRef`` in the repository server CR. + +.. code-block:: yaml + :linenos: + + apiVersion: v1 + kind: Secret + metadata: + name: repository-server-admin + namespace: + type: secrets.kanister.io/kopia-repository/serveradmin + data: + username: + password: + + +TLS Secret +---------- + +This secret stores TLS sensitive data used for Kopia client server communication. +It follows the ``kubernetes.io/tls`` standard. It is referenced by the +``spec.server.tlsSecretRef`` in the repository server CR. + +.. code-block:: yaml + :linenos: + + apiVersion: v1 + kind: Secret + metadata: + name: repository-server-tls + namespace: + type: kubernetes.io/tls + data: + tls.crt: | + + tls.key: | + + + +Repository Server User Access Password Secret +--------------------------------------------- +The Kopia repository client needs an access username and password for authentication to +connect to the Kopia repository server. + +The Kopia client needs a user in the format of ``@``. The username is +the same for all clients, which is specified in ``spec.server.UserAccess.username`` of +the ``RepositoryServer`` CR. The password and host name are provided in the form of +a secret, as shown below: -Controller -========== +.. code-block:: yaml + :linenos: + + apiVersion: v1 + kind: Secret + metadata: + name: repository-server-user-access + namespace: kanister + type: secrets.kanister.io/kopia-repository/serveruser + data: + : + : + + +Kanister Controller +=================== The Kanister controller is a Kubernetes Deployment and is installed easily using ``kubectl``. See :ref:`install` for more information on deploying the controller. @@ -407,3 +849,42 @@ seen by using the following command: Normal Started Phase 23s Kanister Controller Executing phase backupToS3 Normal Update Complete 19s Kanister Controller Updated ActionSet 's3backup-j4z6f' Status->complete Normal Ended Phase 19s Kanister Controller Completed phase backupToS3 + + +Kopia Repository Server Controller +================================== + +The Kopia Repository Server controller is installed as a part of Kanister operator +deployment. See :ref:`deploying_repo_server_controller` for more information on +deploying the controller. + +Execution Walkthrough +--------------------- + +The Repository server controller monitors and responds to create, update, or delete +events for the RepositoryServer custom resource within the same namespace where +it is deployed. When the RepositoryServer CR is created, it sets the ``status.Progress`` +field to ``Pending``. + +The controller starts a Kopia repository server within a Kubernetes pod, using the +configuration provided in the RepositoryServer CR. To access the repository server +within the pod, it also creates a Kubernetes service. This repository server can be +used within the kanister controller as a data mover tool to backup and restore +applications to a Kopia repository. + +.. code-block:: bash + + $ kubectl get pods,svc -n kanister + NAME READY STATUS RESTARTS AGE + pod/kanister-kanister-operator-5b7dfbf97b-5j5p5 2/2 Running 0 33m + pod/repo-server-pod-4tjcw 1/1 Running 0 2m13s + + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + service/kanister-kanister-operator ClusterIP 10.96.197.93 443/TCP 33m + service/repo-server-service-rq2pq ClusterIP 10.96.127.153 51515/TCP 2m13s + +Once the kopia repository server is started successfully,the ``status.Progress`` field of +RepositoryServer CR is set to ``Ready``. + +Refer to :ref:`tutorials` section to understand more about the integration of kanister +and kopia repository server. diff --git a/docs/index.rst b/docs/index.rst index 40274e9a39..4f62c5f9e6 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -21,7 +21,7 @@ and easy to install, operate and scale. overview install - tutorial + kanister_tutorial architecture tasks tooling diff --git a/docs/install.rst b/docs/install.rst index 2af355b6be..7de10bafb1 100644 --- a/docs/install.rst +++ b/docs/install.rst @@ -103,6 +103,22 @@ like below: --set bpValidatingWebhook.tls.caBundle=$(cat /path/to/ca.pem | base64 -w 0) \ --set bpValidatingWebhook.tls.secretName=tls-secret +.. _deploying_repo_server_controller: + +Deploying Kopia Repository Server Controller +============================================ + +To enable the Kopia Repository server controller, set the `repositoryServerController` +value as shown below during the deployment: + +.. substitution-code-block:: bash + + helm upgrade --install kanister kanister/kanister-operator --namespace kanister --create-namespace \ + --set repositoryServerController.enabled=true + +Executing this command will add an additional container to the Kanister pod for the +Kopia repository server controller. + Building and Deploying from Source ================================== diff --git a/docs/kanister_tutorial.rst b/docs/kanister_tutorial.rst new file mode 100644 index 0000000000..ab64d23802 --- /dev/null +++ b/docs/kanister_tutorial.rst @@ -0,0 +1,9 @@ +.. _tutorials: + +Tutorials +********* +.. toctree:: + :maxdepth: 1 + + tutorials/tutorial.rst + tutorials/tutorial_with_repositoryserver.rst diff --git a/docs/overview.rst b/docs/overview.rst index 3114077f0a..44d7564d4b 100644 --- a/docs/overview.rst +++ b/docs/overview.rst @@ -35,7 +35,7 @@ Getting Started Follow the instructions in the :ref:`install` section to get Kanister up and running on your Kubernetes cluster. Then see Kanister in action by going through -the walkthrough under :ref:`tutorial`. +the walkthrough under :ref:`tutorials`. The :ref:`architecture` section provides architectural insights into how things work. We recommend that you take a look at it. diff --git a/docs/templates.rst b/docs/templates.rst index 68352868ca..6c034a6004 100644 --- a/docs/templates.rst +++ b/docs/templates.rst @@ -22,6 +22,7 @@ The TemplateParam struct is defined as: Secrets map[string]v1.Secret Time string Profile *Profile + RepositoryServer *RepositoryServer Options map[string]string Object map[string]interface{} Phases map[string]*Phase @@ -489,7 +490,7 @@ Unlike Secrets and ConfigMaps, only a single profile can optionally be referenced by an ActionSet. As a result, there it is not necessary to name the Profiles in the Blueprint. -The following examples should be helpful. +The following examples should be helpful: .. code-block:: yaml @@ -546,6 +547,54 @@ The currently supported Profile template is based on the following definitions Secret ObjectReference } +RepositoryServers +----------------- + +RepositoryServers are Kanister CustomResource that store information about storage, +repository, and server details. These details are required to run a Kopia repository +server instance, which can then be further used for data operation artifacts. + +In contrast to Secrets and ConfigMaps, an ActionSet can optionally reference only a +single repository server, eliminating the need to name the RepositoryServer in the +Blueprint. + +The following examples should be helpful. + +.. code-block:: yaml + + # Access the RepositoryServer name + "{{ .RepositoryServer.Name }}" + + # Access the RepositoryServer service name + "{{ .RepositoryServer.ServerInfo.ServiceName }}" + +The currently supported RepositoryServer template is based on the following definitions: + +.. code-block:: go + :linenos: + + // RepositoryServer contains fields from Repository server CR that will be used to resolve go templates for repository server in blueprint + type RepositoryServer struct { + Name string + Namespace string + ServerInfo crv1alpha1.ServerInfo + Username string + Credentials RepositoryServerCredentials + Address string + ContentCacheMB int + MetadataCacheMB int + } + + type RepositoryServerCredentials struct { + ServerTLS v1.Secret + ServerUserAccess v1.Secret + } + + type ServerInfo struct { + PodName string `json:"podName,omitempty"` + ServiceName string `json:"serviceName,omitempty"` + } + Options ------- diff --git a/docs/tutorial.rst b/docs/tutorials/tutorial.rst similarity index 99% rename from docs/tutorial.rst rename to docs/tutorials/tutorial.rst index 5aa92b91ce..c607376546 100644 --- a/docs/tutorial.rst +++ b/docs/tutorials/tutorial.rst @@ -1,5 +1,3 @@ -.. _tutorial: - Tutorial ******** diff --git a/docs/tutorials/tutorial_with_repositoryserver.rst b/docs/tutorials/tutorial_with_repositoryserver.rst new file mode 100644 index 0000000000..9fef1a335b --- /dev/null +++ b/docs/tutorials/tutorial_with_repositoryserver.rst @@ -0,0 +1,543 @@ +Using Kopia Repository Server as Data Mover in Blueprint +******************************************************** + +This tutorial will demonstrate how to use Kopia to copy or restore +backups in a Kopia repository. We will be using Kanister functions +that use the Kopia repository Server as the data mover in the blueprint. +For additional documentation on Kanister functions and blueprints, +refer to the :ref:`architecture` and :ref:`kanister functions` +sections, respectively. + +.. contents:: Tutorial Overview + :local: + +Prerequisites +============= + +* For Kubernetes ``1.16`` or higher, you can install Kanister version 0.62.0 + or higher. For cluster version lower than ``1.16``, we recommend installing + Kanister version ``0.62.0`` or lower. + +* `kubectl `_ installed + and setup + +* `helm `_ installed and initialized using the command `helm init` + +* docker + +* The Kopia repository server controller should be deployed along with the Kanister + controller. Refer to the + :ref:`Deploying Kopia Repository server controller ` + section to know more. +* Access to s3 bucket and credentials + +Example Application +=================== + +This tutorial begins by deploying a sample application. While this application is +intentionally simplified, it serves the purpose of demonstrating Kanister's features. +It appends the current time to a log file every second. The application's container +includes the AWS command-line client, which will be described in later sections of +this tutorial. This application is installed in the ``default`` namespace. + +.. code-block:: yaml + + $ cat <> /var/log/time.log; sleep 1; done; truncate /var/log/time.log --size 0; done"] + volumeMounts: + - name: data + mountPath: /var/log + volumes: + - name: data + persistentVolumeClaim: + claimName: time-log-pvc + EOF + +Starting Kopia Repository Server +================================ + +To copy or restore backups to the location storage using the Kopia data mover, +it is necessary to start the Kopia repository server. To learn more about the +Kopia repository server, refer to the :ref:`architecture ` section. + +The repository server controller requires the creation of a Repository Server +custom resource to start the server. To understand more about this custom resource, +see :ref:`architecture`. + +.. _creating_kopia_repository: + +Creating a Kopia Repository +--------------------------- + +The Kopia repository must be created before starting the repository server. + +You can create it as shown below: + +.. code-block:: bash + + $ kopia --log-level=error --config-file=/tmp/kopia-repository.config + --log-dir=/tmp/kopia-cache repository create --no-check-for-updates + --cache-directory=/tmp/cache.dir --content-cache-size-mb=0 --metadata-cache-size-mb=500 + --override-hostname=timelog.app --override-username=kanisterAdmin s3 + --bucket=test-bucket + --prefix=/test/repo-controller + --region=us-east-1 + --access-key= + --secret-access-key= + +To learn more about how to create a repository and gain further insight into the Kopia +repository, refer to the `Kopia documentation `_. + + +Creating Secrets +---------------- + +To learn about the secrets that need to be created for the repository server, +please refer to the :ref:`architecture` section. + +- ``Creating TLS secret`` + +.. code-block:: bash + + $ kubectl create secret tls repository-server-tls-cert --cert=/path/to/certificate.pem --key=/path/to/key.pem -n kanister + +- ``Creating Repository Server User Access Secret`` + +.. code-block:: bash + + $ kubectl create secret generic repository-server-user-access --type='secrets.kanister.io/kopia-repository/serveruser' -n kanister + +- ``Creating Repository Server Admin Secret`` + +.. code-block:: bash + + $ kubectl create secret generic repository-server-admin --type='secrets.kanister.io/kopia-repository/serveradmin' -n kanister --from-literal=username=admin@testpod1 --from-literal=password=test1234 + +- ``Creating Repository Password Secret`` + +.. code-block:: bash + + $ kubectl create secret generic repository-pass --type='secrets.kanister.io/kopia-repository/password' -n kanister --from-literal=repo-password=test1234 + +- ``Creating Storage Location Secret`` + + The secret should contain values identical to the ones used for the ``bucket``, + ``endpoint``, ``region`` fields that were used during the creation of the Kopia + repository. + +.. code-block:: yaml + + $ cat < + # optional: used as a sub path in the bucket for all backups + path: + # optional: specified in case of S3-compatible stores + endpoint: + # required, if supported by the provider + region: + EOF + +- ``Creating Storage Location Credentials Secret`` + +.. code-block:: yaml + + $ cat < + # required: base64 encoded value for the secret corresponding to the key above + secret-acccess-key: + EOF + +.. _creating_repo_server_CR: + +Creating Repository Server Custom Resource +------------------------------------------ + +After creating the secrets, it is necessary to generate a repository server CR that +references the previously created secrets. For more detailed information about the +repository server CR, refer to the :ref:`architecture` section. + +It is important to ensure consistency by using the same values for the fields +``spec.repository.username`` and ``spec.repository.hostname`` in the CR (Custom Resource) +as those used during the repository creation process described in the section +:ref:`Creating a Kopia Repository `. + +The ``--prefix`` field's value is a combination of the prefix specified in +the `spec.data.path` field of the location secret and the sub-path provided in +the ``spec.repository.RootPath`` field of the Repository server CR. + +The ``spec.data.path`` field of the location storage secret ``s3-location`` appended +with the ``spec.repository.RootPath`` in the Repository Server CR should be combined +together to match the ``--prefix`` field of the command used to create the repository, as +specified in the section :ref:`Creating a Kopia Repository `. + + +.. code-block:: yaml + + $ cat < 443/TCP 33m + service/repo-server-service-rq2pq ClusterIP 10.96.127.153 51515/TCP 2m13s + +To verify the successful start of the server, you can use the following command to +check the server's status: + +.. code-block:: bash + + $ kubectl get repositoryservers.cr.kanister.io kopia-repo-server -n kanister -oyaml + apiVersion: cr.kanister.io/v1alpha1 + kind: RepositoryServer + metadata: + annotations: + kubectl.kubernetes.io/last-applied-configuration: | + {"apiVersion":"cr.kanister.io/v1alpha1","kind":"RepositoryServer","metadata":{"annotations":{},"name":"kopia-repo-server","namespace":"kanister"},"spec":{"repository":{"hostname":"timelog.app","passwordSecretRef":{"name":"repository-pass","namespace":"kanister"},"rootPath":"/test/repo-controller","username":"kansiterAdmin"},"server":{"adminSecretRef":{"name":"repository-server-admin","namespace":"kanister"},"tlsSecretRef":{"name":"repository-server-tls-cert","namespace":"kanister"},"userAccess":{"userAccessSecretRef":{"name":"repository-server-user-access","namespace":"kanister"},"username":"kanisteruser"}},"storage":{"credentialSecretRef":{"name":"s3-loc-creds","namespace":"kanister"},"secretRef":{"name":"s3-location","namespace":"kanister"}}}} + creationTimestamp: "2023-06-05T05:45:49Z" + generation: 1 + name: kopia-repo-server + namespace: kanister + resourceVersion: "41529" + uid: b4458c4f-b2d5-4dcd-99de-a0a4d32ed216 + spec: + repository: + hostname: timelog.app + passwordSecretRef: + name: repository-pass + namespace: kanister + rootPath: /test/repo-controller + username: kansiterAdmin + server: + adminSecretRef: + name: repository-server-admin + namespace: kanister + tlsSecretRef: + name: repository-server-tls-cert + namespace: kanister + userAccess: + userAccessSecretRef: + name: repository-server-user-access + namespace: kanister + username: kanisteruser + storage: + credentialSecretRef: + name: s3-loc-creds + namespace: kanister + secretRef: + name: s3-location + namespace: kanister + status: + progress: Ready + serverInfo: + podName: repo-server-pod-4tjcw + serviceName: repo-server-service-rq2pq + +``pod/repo-server-pod-4tjcw`` and ``service/repo-server-service-rq2pq`` populated in +the ``status.serverInfo`` field should be used by the client to connect to the server. + +Invoking Kanister Actions +========================= + +Kanister CustomResources are created in the same namespace as +the Kanister controller. + +The initial Kanister CustomResource to be deployed is referred to as Blueprint. +Blueprints are a set of instructions that direct the controller in executing +actions on an application. An action consists of one or more phases, and each phase +invokes a :doc:`Kanister Function `. Every Kanister function accepts a +string list as input. The ``args`` field in a Blueprint's phase is rendered and passed +into the specified function. + +To learn more about Kanister's CustomResources, see :ref:`architecture`. + +The Blueprint to be created includes two actions, named ``backup`` +and ``restore``. + +The ``backup`` action comprises of a single phase named as +``backupToS3``. ``backupToS3`` invokes the Kanister function ``BackupDataUsingKopiaServer`` +that uses the Kopia repository server to copy backup data to S3 storage. + +The ``restore`` action uses two Kanister functions, ``ScaleWorkload`` and +``RestoreDataUsingKopiaServer``. +The ``ScaleWorkload`` function scales down the ``timelog`` application before +restoring the data. +The ``RestoreDataUsingKopiaServer`` function restores data using the Kopia +repository server from S3 storage. + +To learn more about the Kanister function, refer to the documentation on +:doc:`Kanister's parameter templating `. + +Output artifacts are used in this scenario to store the data path in S3 and +the corresponding snapshot ID, which which will serve as the ``backupIdentifier`` +during the restoration process. + +To know more about artifacts, refer to the :ref:`tutorials` section. + +Blueprint +--------- + +.. code-block:: yaml + + $ cat <> test.log + sh-5.1# cat test.log + hello world + +ActionSet +--------- + +.. code-block:: bash + + # Create action set using the blueprint created in above step + $ kanctl create actionset --action backup --namespace kanister --blueprint time-log-bp --deployment default/time-logger --repository-server kanister/kopia-repo-server + actionset actionset backup-rlcnp created + +The ``--repository-server`` flag is used to provide a reference to the repository server +CR created in step :ref:`Creating Repository Server custom resource `. +Since the CR contains details related to the Kopia repository server and the associated secrets, +the blueprint can access these details using template parameters. This enables the blueprint to +execute backup operations using the Kopia repository server. + + +.. code-block:: bash + + $ kubectl describe actionsets.cr.kanister.io backup-rlcnp -n kanister + + Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal Started Action 14s Kanister Controller Executing action backup + Normal Started Phase 14s Kanister Controller Executing phase backupToS3 + Normal Ended Phase 9s Kanister Controller Completed phase backupToS3 + Normal Update Complete 9s Kanister Controller Updated ActionSet 'backup-rlcnp' Status->complete + + +Let`s delete the date from the ``timelogger`` app. + +.. code-block:: bash + + $ kubectl exec -it time-logger-6d89687cbb-bmdj8 -n default -it sh + sh-5.1# cd /var/log/ + sh-5.1# ls -lrt + total 12 + -rw-r--r-- 1 root root 12 Jun 5 06:22 test.log + -rw-r--r-- 1 root root 7308 Jun 5 06:26 time.log + sh-5.1# rm -rf test.log + sh-5.1# ls -lrt + total 8 + -rw-r--r-- 1 root root 7482 Jun 5 06:26 time.log + + +Now, let's proceed with the restore process by using the ``restore`` action from the +``time-log-bp`` blueprint: + +.. code-block:: bash + + $ kanctl --namespace kanister create actionset --action restore --from "backup-rlcnp" --repository-server kanister/kopia-repo-server + actionset restore-backup-rlcnp-g5h65 create + +The success of the restore operation can be assessed based on the following actionset: + +.. code-block:: bash + + $ kubectl describe actionsets.cr.kanister.io restore-backup-rlcnp-g5h65 -n kanister + + Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal Started Action 20s Kanister Controller Executing action restore + Normal Started Phase 20s Kanister Controller Executing phase shutdownPod + Normal Ended Phase 8s Kanister Controller Completed phase shutdownPod + Normal Started Phase 8s Kanister Controller Executing phase restoreFromS3 + Normal Ended Phase 4s Kanister Controller Completed phase restoreFromS3 + Normal Started Phase 4s Kanister Controller Executing phase bringupPod + Normal Ended Phase 3s Kanister Controller Completed phase bringupPod + Normal Update Complete 2s Kanister Controller Updated ActionSet 'restore-backup-rlcnp-g5h65' Status->complete + +It is necessary to verify that the data has been restored successfully. The presence of +the ``time.log`` file, which was removed prior to the restore process, should confirm +the successful restoration. + +.. code-block:: bash + + $ kubectl exec -it time-logger-6d89687cbb-pv5x6 -n default -it sh + sh-5.1# ls -lrt /var/log + total 16 + -rw-r--r-- 1 root root 12 Jun 5 06:22 test.log + -rw-r--r-- 1 root root 9715 Jun 5 06:32 time.log + sh-5.1# cat /var/log/test.log + hello world + +