Failure while installing trident in Rancher cluster

[Litsegaard]

I'm trying to install v19.04.1 of trident but get the following error:

FATA Install failed; could not check if Trident installer pod is present; pods is forbidden: User "system:serviceaccount:storage:default" cannot list resource "pods" in API group "" at the cluster scope. Resolve the issue; use 'tridentctl uninstall' to clean up; and try again.

What am I doing wrong?

[donny_lang]

What version of Rancher/k8s are you running? 

Can you re-run the installer with the -d flag passed to the installer command to see if there are more verbose logs available?

From a quick look, it appears that the default service account doesn't have the proper view permissions in the namespace. Creating a view-only cluster role and then binding the role to the default service account might be a good way to fix it. Something like: 

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: default-view
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: view
subjects:
  - kind: ServiceAccount
    name: default
    namespace: default

[Litsegaard]

Hi Donny!

Thanks for your reply! "view" was not enough so I decided to "open all valves" and gave the default account "cluster-admin" rights which made the installer go through it's paces, at least a little further:

DEBU Initialized logging.                          logLevel=debug
DEBU Running in a pod, creating API-based client.  namespace=storage
DEBU Initialized Kubernetes API client.            cli=kubectl flavor=k8s namespace=storage version=v1.13.5
DEBU Validated installation environment.           installationNamespace=storage kubernetesVersion=v1.13.5
DEBU Namespace exists.                             namespace=storage
DEBU Parsed YAML into unstructured object.         group=rbac.authorization.k8s.io kind=ClusterRoleBinding version=v1
DEBU Found API resource.                           group=rbac.authorization.k8s.io kind=ClusterRoleBinding resource=clusterrolebindings version=v1
DEBU Deleting object.                              kind=ClusterRoleBinding name=trident-installer namespace=storage
DEBU Object not found for delete, ignoring.        name=trident-installer
DEBU Deleted installer cluster role binding.      
DEBU Parsed YAML into unstructured object.         group=rbac.authorization.k8s.io kind=ClusterRole version=v1
DEBU Found API resource.                           group=rbac.authorization.k8s.io kind=ClusterRole resource=clusterroles version=v1
DEBU Deleting object.                              kind=ClusterRole name=trident-installer namespace=storage
DEBU Object not found for delete, ignoring.        name=trident-installer
DEBU Deleted installer cluster role.              
DEBU Parsed YAML into unstructured object.         group= kind=ServiceAccount version=v1
DEBU Found API resource.                           group= kind=ServiceAccount resource=serviceaccounts version=v1
DEBU Deleting object.                              kind=ServiceAccount name=trident-installer namespace=storage
DEBU Object not found for delete, ignoring.        name=trident-installer
DEBU Deleted installer service account.           
DEBU Parsed YAML into unstructured object.         group= kind=ServiceAccount version=v1
DEBU Found API resource.                           group= kind=ServiceAccount resource=serviceaccounts version=v1
DEBU Creating object.                              kind=ServiceAccount name=trident-installer namespace=storage
DEBU Created object by YAML.                       name=trident-installer
INFO Created installer service account.            serviceaccount=trident-installer
INFO Waiting for object to be created.             objectName=clusterRole
DEBU Parsed YAML into unstructured object.         group=rbac.authorization.k8s.io kind=ClusterRole version=v1
DEBU Found API resource.                           group=rbac.authorization.k8s.io kind=ClusterRole resource=clusterroles version=v1
DEBU Creating object.                              kind=ClusterRole name=trident-installer namespace=storage
DEBU Created object by YAML.                       name=trident-installer
INFO Created installer cluster role.               clusterrole=trident-installer
INFO Waiting for object to be created.             objectName=clusterRoleBinding
DEBU Parsed YAML into unstructured object.         group=rbac.authorization.k8s.io kind=ClusterRoleBinding version=v1
DEBU Found API resource.                           group=rbac.authorization.k8s.io kind=ClusterRoleBinding resource=clusterrolebindings version=v1
DEBU Creating object.                              kind=ClusterRoleBinding name=trident-installer namespace=storage
DEBU Created object by YAML.                       name=trident-installer
INFO Created installer cluster role binding.       clusterrolebinding=trident-installer
DEBU Parsed YAML into unstructured object.         group=rbac.authorization.k8s.io kind=ClusterRoleBinding version=v1
DEBU Found API resource.                           group=rbac.authorization.k8s.io kind=ClusterRoleBinding resource=clusterrolebindings version=v1
DEBU Deleting object.                              kind=ClusterRoleBinding name=trident-installer namespace=storage
DEBU Deleted object by YAML.                       name=trident-installer
INFO Deleted installer cluster role binding.      
DEBU Parsed YAML into unstructured object.         group=rbac.authorization.k8s.io kind=ClusterRole version=v1
DEBU Found API resource.                           group=rbac.authorization.k8s.io kind=ClusterRole resource=clusterroles version=v1
DEBU Deleting object.                              kind=ClusterRole name=trident-installer namespace=storage
DEBU Deleted object by YAML.                       name=trident-installer
INFO Deleted installer cluster role.              
DEBU Parsed YAML into unstructured object.         group= kind=ServiceAccount version=v1
DEBU Found API resource.                           group= kind=ServiceAccount resource=serviceaccounts version=v1
DEBU Deleting object.                              kind=ServiceAccount name=trident-installer namespace=storage
DEBU Deleted object by YAML.                       name=trident-installer
INFO Deleted installer service account.           
FATA Install failed; not implemented.  Resolve the issue; use 'tridentctl uninstall' to clean up; and try again. 

but as you can see I still run into a fatal exception which, unfortunately, isn't very informative. Maybe I should give you some background about how I'm trying to establish trident in my environment.

I'm deploying a Rancher (v2.2.1) custom cluster based on CentOS 7, Docker 18.09.6 (if I remember correctly) and Kubernetes is v1.13.5. Everything works as advertised. In that cluster I've deployed a CentOS 7 based pod with kubectl and a config file making it possible to issue kubectl commands towards the cluster. I've downloaded the trident installer (gz file) which I'm now trying to deploy running in the issues I've described above. I know that there may be some work being done on providing Helm-support for deploying trident in Rancher, but I have to have a look at trident before that solution becomes publicly available.

Cheers

[donny_lang · Accepted solution]

You're right, that log entry is not terribly informative. Is this issue logged on the Trident GitHub repo relevant to your installation environment? 

https://github.com/NetApp/trident/issues/202

[Litsegaard]

Hi Donny!

Thanks for pointing me in the right direction. Providing

--in-cluster=false

solved the problem. Many thanks!!!!

Cheers