Subscribe
Accepted Solution

ONTAP Select : Cannot create

 

Hello,

 

With ONTAP Select Deploy 2.3 and 2.4, I got an error when creating a Cluster. The node cannot be created with the following error.

I tried with Web GUI and CLI.

 

How to solve this issue ?

 

 

ONTAP Select deploy Event :

 

ClusterNodeCreateFailed: One of the node create operations failed during cluster "CLU_TEST" create.

NodeCreateFailed: Node "CLU_TEST-node1" create failed: Cannot create VM 'CLU_TEST-node1' (errType=InvalidRequest).

 

 

sdotadmin_server.log :

 

2017-06-09 09:16:40,046|DEBUG response body: {"code":56,"details":"Cannot create VM 'CLU_TEST-node1' (errType=InvalidRequest)","type":"VmCreateOvfErr"}
response body: {"code":56,"details":"Cannot create VM 'CLU_TEST-node1' (errType=InvalidRequest)","type":"VmCreateOvfErr"}
2017-06-09 09:16:40,060|ERROR |client_api_helper.py|183:vm_create| Error: NodeCreateFailed: Node "CLU_TEST-node1" create failed: Cannot create VM 'CLU_TEST-node1' (errType=InvalidRequest).
|client_api_helper.py|183:vm_create| Error: NodeCreateFailed: Node "CLU_TEST-node1" create failed: Cannot create VM 'CLU_TEST-node1' (errType=InvalidRequest).
2017-06-09 09:16:46,431|DEBUG response body: {"code":13,"details":"<vmname> invalid - no vm named 'CLU_TEST-node1' on host 'nte-esx01.mqt02.mqt'","type":"InvalidArg"}
response body: {"code":13,"details":"<vmname> invalid - no vm named 'CLU_TEST-node1' on host 'nte-esx01.mqt02.mqt'","type":"InvalidArg"}
2017-06-09 09:16:46,844|ERROR |cluster_tasks.py|519:_create_nodes| Cluster [CLU_TEST]: one or more of node create tasks failed, initiating rollback
|cluster_tasks.py|519:_create_nodes| Cluster [CLU_TEST]: one or more of node create tasks failed, initiating rollback
2017-06-09 09:16:46,845|ERROR |cluster_tasks.py|520:_create_nodes| Error: ClusterNodeCreateFailed: One of the node create operations failed during cluster "CLU_TEST" create.
|cluster_tasks.py|520:_create_nodes| Error: ClusterNodeCreateFailed: One of the node create operations failed during cluster "CLU_TEST" create.
2017-06-09 09:16:46,845|ERROR |cluster_tasks.py|277:create_cluster| Cluster [CLU_TEST]: cluster create failed
|cluster_tasks.py|277:create_cluster| Cluster [CLU_TEST]: cluster create failed
2017-06-09 09:16:46,849| INFO |cluster_tasks.py|292:create_cluster| initiating rollback of failed cluster (CLU_TEST)
|cluster_tasks.py|292:create_cluster| initiating rollback of failed cluster (CLU_TEST)
2017-06-09 09:16:46,856|DEBUG |cluster_tasks.py|331:delete_cluster| Delete cluster "CLU_TEST"
|cluster_tasks.py|331:delete_cluster| Delete cluster "CLU_TEST"
2017-06-09 09:16:46,885| INFO |cluster_tasks.py|421:delete_cluster| Cluster [CLU_TEST]: all nodes in cluster are deleted, deleting cluster db
|cluster_tasks.py|421:delete_cluster| Cluster [CLU_TEST]: all nodes in cluster are deleted, deleting cluster db
2017-06-09 09:16:46,886| INFO |cluster_tasks.py|427:delete_cluster| cluster (CLU_TEST) deleted

 

 

 

Regards,

 

Cyrille

Re: ONTAP Select : Cannot create

Is it a standard or distributed vSwitch configured at ESXi level? If distributed, then there is a bug which prevents the deployment and fails at this same level. The workaround is to create a dummy VLAN port group in the standard vSwitch without any nics or any other port groups, and then try the deployment.

 

If standard then perhaps there is some issue which needs to be reviewed.

Re: ONTAP Select : Cannot create

Hello,

 

It was a distributed vSwitch. I change my environment to have supported hardware server. On the new ESX server with standard vSwitch I can create the cluster and deploy the node.

 

Thank you for yout answer and let me know about this bug I will probably have in my customer environment.

 

Cyrille

Re: ONTAP Select : Cannot create

As I understand from NetApp, this bug will be fixed in later releases. In the meantime, as I mentioned you can use the workaround to have the cluster created with a distributed vswitch.

 

Create a dummy test portgroup in the standard vswitch; that should be the only portgroup for the standard vswitch; nothing else. Then you can as usual create your distributed vswitch with nics and port groups and create the cluster. It should work. Post the cluster is created, you can delete the dummy test portgroup from the standard vswitch.

Re: ONTAP Select : Cannot create

Hi @Reg . 

I think I encounter the bug with the Distributed vswitch incompatibility (Ontap Select node deployment fails whatever ontap cluster size/deployment model I choose) . The problem is that all my Vmware cluster are only made of Distributed vswitch so deploying a real standard vswitch instead, with real nics and real port group is not an option for us.

I've read  you're saying that using a "dummy/fake"  standard vswitch (without any physical nics) could be a workaround. But I don't understand exactly what I need to do for that ? Could you please explain a little more the workaround ? 

 

Some precision:

1. my Ontap Deploy VM is hosted on a Vsphere 6.0 cluster, with only distributed vswitch (2 on each cluster)

2. Ontap Select VM are planned to be hosted on 2 different Vsphere 6.0 cluster, with only distributed vswitch (2 on each cluster again, with 2*40Gb nic on each dvs in a LAG LACP Upllink) 

 

Thanks a lot for your help. 

Regards

Re: ONTAP Select : Cannot create

Hi Pierre,

The issue is resolved in deploy version 2.6. Regarding the workaround that I mentioned, all you have to do is just create a standard vswitch, using the create option, and follow the default options it gives you, without having to add any portgroups, vmkernel or any nics, etc.

Also, before you create the cluster, are you able to do the networking test "network connectivity-test" from the CLI. This option I believe is there in the GUI too with deploy version 2.6

 

regards,

Reg

Re: ONTAP Select : Cannot create

Hi @Reg

 

I've tried with Ontap Select/Deploy 2.6 without any more luck or better result. The cluster creation fails at the same moment with same message : "... create failed. Reason: Error during configuration of vnics:  InvalidRequestFault."

 

I've checked log files:

1. sdotadmin-server.log does not give me any more information about the cause

 

2. esxadmin.log gives me this:

 

[INFO] [Thu Nov 23 05:43:31 2017] [8815] Configuring Network ...
[INFO] [Thu Nov 23 05:43:31 2017] [8815] Error encountered while creating VM
[INFO] [Thu Nov 23 05:43:31 2017] [8815] [13] InvalidArg: <scsiC:N> invalid - Invalid/missing disk 'scsi0:0'
[ERROR] [Thu Nov 23 05:43:31 2017] [8815] [51] Err: Error during configuration of vnics: InvalidRequestFault

 

 

I've tested with external storage (SAN) disks and with local DAS disks with the same failure. I've also tested each run (external vs local DAS) with 2 node HA cluster and 1 node ciuster (both in evaluation mode) with the same failure. 

I've tested on 2 different cluster, but both configured with 1 Distributed vswith and 2 distributed port group (one for external and one for internal) , as well as with the same 2*40Gb LAG LACP uplink) 

And I've also ran a "Ontap network connectivity check" (in quick mode) which succeeded. 

 

 

All these failure may be related to my Vmware infrastructure but as I'm time constrained, I don't have much time to troubleshoot with Netapp . Also, I do not want to make any modification to my Vmware cluster as I think that it's up to Ontap Select Appliance to adapt, in a transparent way, to the underlying Vmware hosting infrastructure. Not the opposite.

 

So i'm will give up with Ontap Select and use a more robust and mature software solution (like a much more classical linux box with smb + nfs + drbd + pacemaker). I will loose some interested features (especially snapmirror) but i will save much time (as well as money) and could also bypass wanted feature with technical alternatives. 

 

Thanks anyway for taking time to help me. 

 

Regards, Pierre.

Re: ONTAP Select : Cannot create

Hi Pierre,

Perhaps I could logon myself and assist you. Is the sdotadmin-server.log reporting the same errors as previous when you initially raised the case? Also, are you able to provide the commands that you have executed for local DAS and for external disk array? And screenshots of the local DAS datastores and external array datastores.

 

I guess the problem is with it not being able to recognise the disk, which could be an issue with the configuration or commands you run.

 

ONTAP Select is good, depending on what you want to achieve and what feature sets you want. If it is normal file sharing capabilities, snapshots and replications there are other SDS offering from different vendors too, with their own pros and cons.

 

regards,

Reg

Re: ONTAP Select : Cannot create

@Reg

"Is the sdotadmin-server.log reporting the same errors as previous when you initially raised the case?"

==> Yes

 

 

"Also, are you able to provide the commands that you have executed for local DAS and for external disk array? And screenshots of the local DAS datastores and external array datastores."

==> Sorry but don't have any screenshots. And no command either as I used OntapDeploy GUI to create cluster. 

But I can send you auto support if you want to investigate a little bit more. 

 

 

I'm sure ontap select is good and works as expected in some shops. The problem is that it seems incompatible in my technical environment. We own many Netapp filer for a long long times and have a strong and historical relationship with Netapp and I know that it will probably consume much more time to make Ontap Select work in our technical environment  than  use other solution.

It is a very pragmatic approach justified by time constraint I have on the project I'm working on. 

 

Regards. Pierre. 

 

Re: ONTAP Select : Cannot create

OK understand. Send me the autosupport and screenshot of the datastore or disk that you are pointing to scsi0:0 to be used as the storage pool. Within vSphere client--> configuration-->storage->datastore

 

Show me the output from the deploy command line

host show-all

license show-all

cluster show-all

 

Assuming you are setting up a multi-node cluster (2 node cluster), then setup the cluster using command line. Would you option to use local within the ESXi? If so, that will be easy, as external storage is only supported for single-node.

 

These is an example of commands. Follow the exact syntax. Note the double \ in the username. I've put FQDN DNS names. You will have to create DNS entries for the below

OTSnode1.com.org to OTSnode2.com.org are your ESXi physical hosts

OTScluster01 --> OTS cluster name

otsclusn01 --> OTS VM residing on ESXi node1

otsclusn02 --> OTS VM residing on ESXi node2

 

host add --host-id OTSnode1.com.org --username <domainname\\domainusername> --password Pass@#123 --vcenter vCenter.com.org

host add --host-id OTSnode2.com.org --username <domainname\\domainusername> --password Pass@#123 --vcenter vCenter.com.org

 

 

Run the host show-all command after each above command to verify the entry is created. The status should show as authenticated.

 

If you are using eval licenses, and want to conifgure a standard configuration of OTS that uses 4vCPU and 16 GB RAM, then the small instance type is selected below, else for premium which is v8CPU and 64GB perhaps requires license and instance type will be medium

Replace Location1 with the site under which the ESXi hosts are registered under in vCenter. Storage pool name is the name of the datastore as you see in vCenter. Replace the internal-network and externale network names with the portgroups you have created in the distributed vSwitch. OTS Management network is usually the external network, unless you have created a seperate one for it.

 

host configure --host-id OTSnode1.com.org --location Location1 --storage-pool DSotsnode01 --eval --internal-network ONTAP-Internal --management-network ONTAP-External --data-network ONTAP-External --instance-type small

host configure --host-id OTSnode2.com.org --location Location1 --storage-pool DSotsnode02 --eval --internal-network ONTAP-Internal --management-network ONTAP-External --data-network ONTAP-External --instance-type small

 

 

Run the host show-all command after each above command to verify the entry. The status should change from authenticated to configured.

You can run the license show-all command too to verify output

 

Run the network connectivity check prior to running the cluster create

 

Finally the cluster create command as below which will install ontap 9.2

 

cluster create --name OTScluster01 --admin-password netapp123 --cluster-mgmt-ip 10.118.111.103 --netmask 255.255.255.224 --gateway 10.118.111.97 --ontap-image-version 9.2 --node-names otsclusn01 otsclusn02 --node-mgmt-ips 10.118.111.101 10.118.111.102 --node-hosts OTSnode1.com.org OTSnode2.com.org --node-mirrors otsclusn02 otsclusn01

 

run the cluster show-all command to verify status.. The same status can be checked through the GUI.