Management Server fails to start on new install

[iloving]

problem

Followed the instructions to create a new management server with a separate mysql server. Like many others, the server fails to start. When I try to go to http::8080, I get:

HTTP ERROR 503 Service Unavailable

URI:	/client/
503
Service Unavailable

When I look at the logs, I see multiple errors, although the two that stick out for me are:
management-server.out: lots of

2025-09-05 22:24:28,011 main ERROR TcpSocketManager (TCP:localhost:4560) caught exception and will continue: java.io.IOException: Unable to create socket for localhost at port 4560 using ip address 127.0.0.1 and port 4560

management-server.log:
Unhandled configuration exception: The script injectkeys.sh failed to run successfully : Files /var/lib/cloudstack/management/.ssh/id_rsa and /usr/share/cloudstack-common/scripts/vm/systemvm/id_rsa.cloud differsudo: a password is required

2025-09-05 22:26:17,543 DEBUG [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:[]) (logid:) module start failure of module [core] was due to: org.springframework.context.ApplicationContextException: Failed to start bean 'cloudStackLifeCycle'; nested exception is com.cloud.utils.exception.CloudRuntimeException: Unhandled configuration exception

tons of module load errors for various modules such as:
2025-09-05 22:26:17,604 WARN [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:[]) (logid:) Failed to start module [allocator] due to: [Error creating bean with name 'org.springframework.beans.factory.config.PropertyPlaceholderConfigurer#0' defined in URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.21.0.0.jar!/META-INF/cloudstack/bootstrap/spring-bootstrap-context-inheritable.xml]: Cannot resolve reference to bean 'DefaultConfigResources' while setting bean property 'locations'; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'DefaultConfigResources' available].

This appears to be the same problem someone else had, but it doesn't look like it went anywhere. #11534 (reply in thread)

versions

ACS: v4.21
mysql: 8.0.43-0ubuntu0.22.04.1
Running on a Ubuntu 22.04 under VMWare Cloud.
1 nic

The steps to reproduce the bug

1. Install the server as per instructions

What to do about it?

Cloudstack is currently unusable out of the box.

I have attached:
-all the logs generated by the server
-a screen capture of the terminal
-the script I use to do the installation of cloudstack

screenshot.txt

install-cloudstack1.sh

logs.tgz

[rastopato]

I have exact same problem. I have multiple environments. It happened on new install, but also on upgrade of production one. Only way back was restore database and ACS management server.
The same error: Cannot resolve reference to bean 'DefaultConfigResources' while setting bean property 'locations'.
It looks like upgrade from 4.20.1 to 4.21 cause some error in database.
I did upgrades multiple times successfully on the same environment from 4.19.1. This one causing the issue.

[iloving]

I don't think deployment onto Ubuntu was tested at all, because as a test I set up a Rocky 9 instance and was able to install Cloudstack without issue on that.

[rastopato]

I have it running on RHEL 9.6.
Database Mysql 8.x on separate RHEL 9.6.

[weizhouapache]

@rastopato
the issue is caused by

sudo: a password is required

can you update sudoers file to allow cloud user to run commands without password prompt ?

for example

cloud ALL=(ALL) NOPASSWD:ALL

or update the file /etc/sudoers.d/cloudstack-management

[rastopato]

ok, very first error is:
2025-09-08 17:33:19,365 ERROR [c.c.u.d.ScriptRunner] (main:[]) (logid:) Error executing: ALTER TABLE cloud.kubernetes_cluster ADD CONSTRAINT fk_cluster__control_node_service_offering_id FOREIGN KEY fk_cluster__control_node_service_offering_id(control_node_service_offering_id) REFERENCES service_offering(id) ON DELETE CASCADE
2025-09-08 17:33:19,369 ERROR [c.c.u.d.ScriptRunner] (main:[]) (logid:) java.sql.SQLException: Duplicate foreign key constraint name 'fk_cluster__control_node_service_offering_id'
2025-09-08 17:33:19,373 ERROR [c.c.u.DatabaseUpgradeChecker] (main:[]) (logid:) Unable to execute upgrade script java.sql.SQLException: Duplicate foreign key constraint name 'fk_cluster__control_node_service_offering_id'

[iloving]

The missing sudoers entry appears to have been the blocker for me. The management server now installs and runs.

I guess it got missed in the ubuntu install packaging.

[weizhouapache]

ok, very first error is: 2025-09-08 17:33:19,365 ERROR [c.c.u.d.ScriptRunner] (main:[]) (logid:) Error executing: ALTER TABLE cloud.kubernetes_cluster ADD CONSTRAINT fk_cluster__control_node_service_offering_id FOREIGN KEY fk_cluster__control_node_service_offering_id(control_node_service_offering_id) REFERENCES service_offering(id) ON DELETE CASCADE 2025-09-08 17:33:19,369 ERROR [c.c.u.d.ScriptRunner] (main:[]) (logid:) java.sql.SQLException: Duplicate foreign key constraint name 'fk_cluster__control_node_service_offering_id' 2025-09-08 17:33:19,373 ERROR [c.c.u.DatabaseUpgradeChecker] (main:[]) (logid:) Unable to execute upgrade script java.sql.SQLException: Duplicate foreign key constraint name 'fk_cluster__control_node_service_offering_id'

@rastopato
I guess it is not the first try, right ?
when you try to upgrade again, you need to restore the db to the original state before upgrade

[rastopato]

ok, very first error is: 2025-09-08 17:33:19,365 ERROR [c.c.u.d.ScriptRunner] (main:[]) (logid:) Error executing: ALTER TABLE cloud.kubernetes_cluster ADD CONSTRAINT fk_cluster__control_node_service_offering_id FOREIGN KEY fk_cluster__control_node_service_offering_id(control_node_service_offering_id) REFERENCES service_offering(id) ON DELETE CASCADE 2025-09-08 17:33:19,369 ERROR [c.c.u.d.ScriptRunner] (main:[]) (logid:) java.sql.SQLException: Duplicate foreign key constraint name 'fk_cluster__control_node_service_offering_id' 2025-09-08 17:33:19,373 ERROR [c.c.u.DatabaseUpgradeChecker] (main:[]) (logid:) Unable to execute upgrade script java.sql.SQLException: Duplicate foreign key constraint name 'fk_cluster__control_node_service_offering_id'

@rastopato I guess it is not the first try, right ? when you try to upgrade again, you need to restore the db to the original state before upgrade

It was from restored to original working state. I don't have time to try it again now, but later I will try and collect all logs. But I didn't have success 3 times on 2 different environments :(