The plugins are part of the agent so no agent = no plugins.
In SC 3.x we used to have a built in local agent which would bypass our real agent if you didnt want to use one, however since we moved to java this is no longer possible so as of 4.0 an agent is required to do anything with a plugin.
We are coming out with a new agent in SC 4.1 which is built for scalability, the 4.0 agent is same agent we always had which works great but it is only single threaded. Meaning you can only do one backup at a time, if you get multiple parallel backups things wont work well. If this is case you probably should stay on SC 3.6P1 until SC 4.1 is available due to restriction in 4.0 of having the agent.
Regards,
Keith