topic Re: Failed to catalog (nas2) - Storage layout could not be determined on Snapcenter in Data Protection

Failed to catalog (nas2) - Storage layout could not be determined on Snapcenter

Chiatan — Wed, 04 Jun 2025 14:56:33 GMT

Hello Gent's,

Any one has seen this error while backingup the volume ? looks it's looking for a log, which is not present.

hought to share few more logs from snapcenter :

Protection relationship created from Primary to Archive

Protection relationship initialized from Primary to Archive

Protection Job completed

Update catalog on Archive : vol2

( Job 119 ) Catalog of backup 'vol2_SnapCenter_20170621060323_daily_116' (

Job 120 ) Catalog CIFS share '\\NAS2\share2'

( Job 121 ) Mount snapshot 'vol2_SnapCenter_20170621060323_daily_116' of share '\\NAS2\share2'

( Job 122 ) stop Maintenance

( Job 118 ) Create Resource Group for Volume vol2

______

Storage layout could not be determined. Please check the Windows event log and SnapDrive logs for more details.

Failed to catalog (nas2) - Storage layout could not be determined.

Please check the Windows event log and SnapDrive logs for more details.

2017-06-20T17:44:26.0000666+01:00 ERROR qtp1728427875-23 c.n.s.l.u.Zipper - Zipper : Unable to visit File /var/opt/snapcenter/catalog/logs/catalog_16.log

Looks it's trying to access the catalog_16.log, as there is no file..

logs]# ls -al
total 12
drwxr-xr-x. 2 root root 70 Jun 20 17:40 .
drwxrwxr-x. 4 root root 27 Jun 20 16:04 ..
-rw-r--r--. 1 root root 203 Jun 20 17:40 catalog_0.log
-rw-r--r--. 1 root root 3680 Jun 20 17:40 catalog_15.log
-rw-r--r--. 1 root root 148 Jun 20 16:04 catalog_init.log

java.nio.file.NoSuchFileException: /var/opt/snapcenter/catalog/logs/catalog_16.log at sun.nio.fs.UnixException.translateToIOException(Unknown Source) ~[na:1.8.0_131] at sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source) ~[na:1.8.0_131] at sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source) ~[na:1.8.0_131] at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(Unknown Source) ~[na:1.8.0_131] at sun.nio.fs.UnixFileSystemProvider.readAttributes(Unknown Source) ~[na:1.8.0_131] at sun.nio.fs.LinuxFileSystemProvider.readAttributes(Unknown Source) ~[na:1.8.0_131] at java.nio.file.Files.readAttributes(Unknown Source) [na:1.8.0_131] at java.nio.file.FileTreeWalker.getAttributes(Unknown Source) ~[na:1.8.0_131] at java.nio.file.FileTreeWalker.visit(Unknown Source) ~[na:1.8.0_131] at java.nio.file.FileTreeWalker.walk(Unknown Source) ~[na:1.8.0_131] at java.nio.file.Files.walkFileTree(Unknown Source) [na:1.8.0_131] at com.netapp.smcore.loader.utils.Zipper.zip(Zipper.java:99) [spl-main-2.0.jar:na] at com.netapp.catalog.service.CatalogLogDumpService.getJobLogs(CatalogLogDumpService.java:65) [catalog-2.0.jar:na] at com.netapp.catalog.service.CatalogLogDumpService.getLogs(CatalogLogDumpService.java:111) [catalog-2.0.jar:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_131] at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_131] at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_131] at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_131] at com.netapp.smcore.loader.utils.PluginUtils.executePluginMethodOfPluginClass(PluginUtils.java:82) [spl-main-2.0.jar:na] at
com.netapp.smcore.loader.utils.PluginUtils.executePluginMethodOfPluginClass(PluginUtils.java:133) [spl-main-2.0.jar:na] at com.netapp.smcore.loader.resource.SMcoreResourceManager.handleAllRequests(SMcoreResourceManager.java:588) [spl-main-2.0.jar:na] at com.netapp.smcore.loader.resource.SMcoreResourceManager.dumpLogs(SMcoreResourceManager.java:225) [spl-main-2.0.jar:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_131] at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_131] at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_131] at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_131] at com.netapp.smcore.loader.startup.AuthenticationProxy.invoke(AuthenticationProxy.java:28) [spl-main-2.0.jar:na] at com.sun.proxy.$Proxy17.dumpLogs(Unknown Source) [na:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_131] at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_131] at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_131] at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_131] at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:188) [cxf-bundle-jaxrs-2.7.11.jar:2.7.11] at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:104) [cxf-bundle-jaxrs-2.7.11.jar:2.7.11] at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:204) [cxf-bundle-jaxrs-2.7.11.jar:2.7.11] at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:101) [cxf-bundle-jaxrs-2.7.11.jar:2.7.11] at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58) [cxf-bundle-jaxrs-2.7.11.jar:2.7.11] at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:94) [cxf-bundle-jaxrs-2.7.11.jar:2.7.11] at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:272) [cxf-bundle-jaxrs-2.7.11.jar:2.7.11] at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121) [cxf-bundle-jaxrs-2.7.11.jar:2.7.11] at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.serviceRequest(JettyHTTPDestination.java:355) [cxf-bundle-jaxrs-2.7.11.jar:2.7.11] at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:319) [cxf-bundle-jaxrs-2.7.11.jar:2.7.11] at org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:65) [cxf-bundle-jaxrs-2.7.11.jar:2.7.11] at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1088) [jetty-server-8.1.14.v20131031.jar:8.1.14.v20131031] at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1024) [jetty-server-8.1.14.v20131031.jar:8.1.14.v20131031] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) [jetty-server-8.1.14.v20131031.jar:8.1.14.v20131031] at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) [jetty-server-8.1.14.v20131031.jar:8.1.14.v20131031] at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) [jetty-server-8.1.14.v20131031.jar:8.1.14.v20131031] at org.eclipse.jetty.server.Server.handle(Server.java:370) [jetty-server-8.1.14.v20131031.jar:8.1.14.v20131031] at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) [jetty-server-8.1.14.v20131031.jar:8.1.14.v20131031] at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:971) [jetty-server-8.1.14.v20131031.jar:8.1.14.v20131031] at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1033) [jetty-server-8.1.14.v20131031.jar:8.1.14.v20131031] at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:651) [jetty-http-8.1.14.v20131031.jar:8.1.14.v20131031] at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) [jetty-http-8.1.14.v20131031.jar:8.1.14.v20131031] at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) [jetty-server-8.1.14.v20131031.jar:8.1.14.v20131031] at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:196) [jetty-io-8.1.14.v20131031.jar:8.1.14.v20131031] at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667) [jetty-io-8.1.14.v20131031.jar:8.1.14.v20131031] at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) [jetty-io-8.1.14.v20131031.jar:8.1.14.v20131031] at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) [jetty-util-8.1.14.v20131031.jar:8.1.14.v20131031] at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) [jetty-util-8.1.14.v20131031.jar:8.1.14.v20131031] at java.lang.Thread.run(Unknown Source) [na:1.8.0_131]

Logs from Index server:

2017-06-20T17:40:31.0000952+01:00 INFO qtp1728427875-25 o.a.z.ZooKeeper - Initiating client connection, connectString=poc.mtest.co.uk:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$3@2c31e8d7
2017-06-20T17:40:31.0000978+01:00 INFO qtp1728427875-25-SendThread(poc.mtest.co.uk:2181) o.a.z.ClientCnxn - Opening socket connection to server poc.mtest.co.uk/10.1.1.1:2181. Will not attempt to authenticate using SASL (unknown error)
2017-06-20T17:40:31.0000981+01:00 INFO qtp1728427875-25-SendThread(poc.mfltest.co.uk:2181) o.a.z.ClientCnxn - Socket connection established to poc.mtest.co.uk/10.1.1.1:2181, initiating session
2017-06-20T17:40:32.0000001+01:00 INFO qtp1728427875-25-SendThread(poc.mtest.co.uk:2181) o.a.z.ClientCnxn - Session establishment complete on server poca.mfltest.co.uk/10.1.1.1:2181, sessionid = 0x15cc5faf9950008, negotiated timeout = 10000
2017-06-20T17:40:32.0000355+01:00 DEBUG qtp1728427875-25 c.n.c.CatalogBackup - Cataloged backup object 'nas2_SnapCenter_20170620173734_daily_14' having Id '45816470-99a7-42c0-abd4-fa03521f2759', SVM 'vserver2', Volume 'nas2', Path Count '0'
2017-06-20T17:40:32.0000355+01:00 DEBUG qtp1728427875-25 c.n.c.s.CatalogService - Exit registerBackup

So is it right to understand port 2181 is not open ? and if spl is installed correclty, there should be some service using that port !!!!!

Thanks

Chaitan

Re: Failed to catalog (nas2) - Storage layout could not be determined on Snapcenter

Jeff_Yao — Wed, 28 Jun 2017 05:57:22 GMT

looks like it's not about the port.

try below to see if helps

It sounds like the catalog is not completely up and running. You might try the following:

Catalog server connection tips.
1. Is the firewall daemon on each of the catalog servers stopped and disabled
2. Are the following ports open and available?
8145, 2181, 2888, 3888, 8983
3. Can you ping each of the catalog servers from the SnapCenter server?
4. Can you ping the SnapCenter server from each of the catalog servers?

Steps to verify correct installation of index server
From index server, run
ps -aux | grep zookeeper
ps -aux | grep solr
to make sure these services are running
Or you can browse to the Solr server using http://<index_server_IP>:8983 to see the status

5. Check all the index server processes are working OK using netstat –tn to see if the ports have listeners)

How to stop/start index server processes
How to stop
/opt/NetApp/snapcenter/indexer/solr-6.2.0/bin/solr stop -p 8983
/opt/NetApp/snapcenter/indexer/zookeeper-3.4.9/bin/zkServer.sh stop
How to start
/opt/NetApp/snapcenter/indexer/solr-6.2.0/bin/solr start -c -z<index_server_IP>:2181,<snapcenter_server>:2181

The following might also be helpful:

Steps to manually re-index a snapshot

If a backup completes but does not get catalog for some reason you can force SnapCenter to recatalog that backup. there is a PowerShell cmdlet for this:

New-SmBackupCatalog -Backup <backup_name> -PerformCompleteCatalog -Verbose

Backup name should be the baseline snapshot name created.

Re: Failed to catalog (nas2) - Storage layout could not be determined on Snapcenter

Chiatan — Wed, 28 Jun 2017 06:49:59 GMT

Thanks for your response.

Just tired this :

[root@cgpocalt01 tmp]# /opt/NetApp/snapcenter/indexer/solr-6.2.0/bin/solr stop -p 8983
Sending stop command to Solr running on port 8983 ... waiting 5 seconds to allow Jetty process 5124 to stop gracefully.

[root@cgpocalt01 tmp]# /opt/NetApp/snapcenter/indexer/zookeeper-3.4.9/bin/zkServer.sh stop
ZooKeeper JMX enabled by default
Using config: /opt/NetApp/snapcenter/indexer/zookeeper-3.4.9/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED

[root@cgpocalt01 tmp]mo/opt/NetApp/snapcenter/indexer/solr-6.2.0/bin/solr start -c -z 10.10.108.128:2181,10.10.108.126:2181 -f

Starting Solr in SolrCloud mode on port 8983 from /opt/NetApp/snapcenter/indexer/solr-6.2.0/server

0 INFO (main) [ ] o.e.j.u.log Logging initialized @12026ms
214 INFO (main) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
228 INFO (main) [ ] o.e.j.d.p.ScanningAppProvider Deployment monitor [file:///opt/NetApp/snapcenter/indexer/solr-6.2.0/server/contexts/] at interval 0
791 INFO (main) [ ] o.e.j.w.StandardDescriptorProcessor NO JSP Support for /solr, did not find org.apache.jasper.servlet.JspServlet
800 WARN (main) [ ] o.e.j.s.SecurityHandler ServletContext@o.e.j.w.WebAppContext@20398b7c{/solr,file:///opt/NetApp/snapcenter/indexer/solr-6.2.0/server/solr-webapp/webapp/,STARTING}{/opt/NetApp/snapcenter/indexer/solr-6.2.0/server/solr-webapp/webapp} has uncovered http methods for path: /
807 INFO (main) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): WebAppClassLoader=1262822392@4b4523f8
844 INFO (main) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
844 INFO (main) [ ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /opt/NetApp/snapcenter/indexer/solr-6.2.0/server/solr
845 INFO (main) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/opt/NetApp/snapcenter/indexer/solr-6.2.0/server/solr'
845 INFO (main) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
845 INFO (main) [ ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /opt/NetApp/snapcenter/indexer/solr-6.2.0/server/solr
857 INFO (main) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
892 INFO (main) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
16066 WARN (main-SendThread(10.10.108.128:2181)) [ ] o.a.z.ClientCnxn Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)

31274 ERROR (main) [ ] o.a.s.s.SolrDispatchFilter Could not start Solr. Check solr/home property and the logs
31324 ERROR (main) [ ] o.a.s.c.SolrCore null:org.apache.solr.common.SolrException: Error occurred while loading solr.xml from zookeeper
at org.apache.solr.servlet.SolrDispatchFilter.loadNodeConfig(SolrDispatchFilter.java:184)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:156)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:134)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)

31325 INFO (main) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
31342 INFO (main) [ ] o.e.j.s.h.ContextHandler Started o.e.j.w.WebAppContext@20398b7c{/solr,file:///opt/NetApp/snapcenter/indexer/solr-6.2.0/server/solr-webapp/webapp/,AVAILABLE}{/opt/NetApp/snapcenter/indexer/solr-6.2.0/server/solr-webapp/webapp}
31350 INFO (main) [ ] o.e.j.s.ServerConnector Started ServerConnector@f8c1ddd{HTTP/1.1,[http/1.1]}{0.0.0.0:8983}
31350 INFO (main) [ ] o.e.j.s.Server Started @43378ms

Thanks

Chaitan

Re: Failed to catalog (nas2) - Storage layout could not be determined on Snapcenter

Jeff_Yao — Wed, 28 Jun 2017 07:10:23 GMT

then probably better to open a case to resolve it?