# SES 3 installation issues **URL:** \ **Category:** SUSE Enterprise Storage\ **Created:** [June 15, 2016, 1:38pm UTC](https://forums.suse.com/t/ses-3-installation-issues/28287 "2016-06-15T13:38:38Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![polezhaevdmi](https://avatars.discourse-cdn.com/v4/letter/p/ccd318/32.png) [@polezhaevdmi](https://forums.suse.com/u/polezhaevdmi)\ **Post date:** [June 15, 2016, 1:38pm UTC](https://forums.suse.com/t/ses-3-installation-issues/28287/1 "2016-06-15T13:38:38Z")
Thread for marking and discussing SUSE Enterprise Storage issues in a process of installation and supplemental manuals also. Link for the quick manual is [here](https://www.suse.com/documentation/ses-1/book_storage_admin/data/ceph_install_ceph-deploy.html) 2.1.2 [Preparing Each Ceph Node](https://www.suse.com/documentation/ses-1/book_storage_admin/data/ceph_install_ceph-deploy.html#ceph_install_ceph-deploy_eachnode) > [@](#): > > Install NTP. NTP already installed. > [@](#): > > Install SSH server. OpenSSH already installed and running. > [@](#): > > Add a ceph user. ses01:~ # useradd -m ceph useradd: user ‘ceph’ already exists > [@](#): > > 6. On the admin node, become the ceph user, ses01:~ # su - ceph This account is currently not available. The user ceph exists, but with login disabled and nologin shell set. Solution: run YAST2, go to “User and Group Management”, filter system users, edit user “ceph”, remove “Disable User login” mark and set login shell to /bin/bash. > [@](#): > > Copy the key to each Ceph node: ceph@ses01:~\> ssh-copy-id [ceph@172.18.66.16](mailto:ceph@172.18.66.16) The authenticity of host ‘172.18.66.16 (172.18.66.16)’ can’t be established. ECDSA key fingerprint is 04:55:4b:a6:42:b4:0f:db:86:b9:4b:3b:c0:ab:5e:5f [MD5]. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed – if you are prompted now it is to install the new keys Password: Password: Password: Permission denied (publickey,keyboard-interactive). How to resolve that in better way, and why this is appeared if everything installed just out of the box? ---
**Author:** ![polezhaevdmi](https://avatars.discourse-cdn.com/v4/letter/p/ccd318/32.png) [@polezhaevdmi](https://forums.suse.com/u/polezhaevdmi)\ **Post date:** [June 15, 2016, 4:39pm UTC](https://forums.suse.com/t/ses-3-installation-issues/28287/2 "2016-06-15T16:39:24Z")
> [@](#): > > How to resolve that in better way Solution: run YAST2, go to “User and Group Management”, filter system users, edit user “ceph”, set the unified known password and use this password when prompted from ssh-copy-id utility. ---
**Author:** ![polezhaevdmi](https://avatars.discourse-cdn.com/v4/letter/p/ccd318/32.png) [@polezhaevdmi](https://forums.suse.com/u/polezhaevdmi)\ **Post date:** [June 15, 2016, 6:15pm UTC](https://forums.suse.com/t/ses-3-installation-issues/28287/3 "2016-06-15T18:15:50Z")
```auto Run ceph-deploy to install Ceph on each node ``` [I]… [ceph\_deploy.install][DEBUG] Detecting platform for host ses02 … [ses02][DEBUG] connection detected need for sudo sudo: no tty present and no askpass program specified [ses02][DEBUG] connected to host: ses02 [ceph\_deploy][ERROR] RuntimeError: remote connection got closed, ensure `requiretty` is disabled for ses02[/I] Solution: DO NOT use visudo and add “Defaults:ceph !requiretty” line, as written in the [Ceph quick-deploy book](http://docs.ceph.com/docs/master/start/quick-ceph-deploy/) and SES manual! Use YaST2 instead, “Security and Users”, “Sudo” and add user ceph to run all commands from root without password. I dont know why the direct editing via visudo did not affected the system, but YaST2 made all changes fine. ---
**Author:** ![polezhaevdmi](https://avatars.discourse-cdn.com/v4/letter/p/ccd318/32.png) [@polezhaevdmi](https://forums.suse.com/u/polezhaevdmi)\ **Post date:** [June 15, 2016, 6:35pm UTC](https://forums.suse.com/t/ses-3-installation-issues/28287/4 "2016-06-15T18:35:53Z")
> [@](#): > > Install the client part of Calamari Already installed in the system. [SIZE=1][/SIZE]Loading repository data… Reading installed packages… ‘calamari-clients’ not found in package names. Trying capabilities. ‘romana’ providing ‘calamari-clients’ is already installed. Resolving package dependencies… Nothing to do. ---
**Author:** ![smflood](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/smflood/32/10576_2.png) [@smflood](https://forums.suse.com/u/smflood)\ **Post date:** [June 15, 2016, 6:57pm UTC](https://forums.suse.com/t/ses-3-installation-issues/28287/5 "2016-06-15T18:57:38Z")
On 15/06/16 11:44, polezhaevdmi wrote: [color=blue] > Thread for marking and discussing SUSE Enterprise Storage issues in a > process of installation and supplemental manuals also. > > Link for the quick manual is ‘here’ > ([https://www.suse.com/documentation/ses-1/book\_storage\_admin/data/ceph\_install\_ceph-deploy.html](https://www.suse.com/documentation/ses-1/book_storage_admin/data/ceph_install_ceph-deploy.html)) > > 2.1.2 ‘Preparing Each Ceph Node’ > ([https://www.suse.com/documentation/ses-1/book\_storage\_admin/data/ceph\_install\_ceph-deploy.html#ceph\_install\_ceph-deploy\_eachnode](https://www.suse.com/documentation/ses-1/book_storage_admin/data/ceph_install_ceph-deploy.html#ceph_install_ceph-deploy_eachnode))[/color] Note the above URLs are for SUSE Enterprise Storage version 1.0 documentation and not version 3.0 that your subject line indicates. For SES 3.0 you should follow [https://www.suse.com/documentation/ses-3/book\_storage\_admin/data/ceph\_install\_ceph-deploy.html](https://www.suse.com/documentation/ses-3/book_storage_admin/data/ceph_install_ceph-deploy.html) [color=blue][color=green] > > Install NTP.[/color] > > NTP already installed. > > [color=green] > > Install SSH server.[/color] > > OpenSSH already installed and running. > > [color=green] > > Add a ceph user.[/color] > > -ses01:~ # useradd -m ceph > > useradd: user ‘ceph’ already exists- > > [color=green] > > 6. On the admin node, become the ceph user,[/color] > > -ses01:~ # su - ceph > > This account is currently not available.- > > The user ceph exists, but with login disabled and nologin shell set. > > Solution: run YAST2, go to “User and Group Management”, filter system > > users, edit user “ceph”, remove “Disable User login” mark and set login > > shell to /bin/bash.[/color] If you are following SES 1.0 documentation (which the above steps would suggest) this would explain your difficulties. ## HTH. Simon SUSE Knowledge Partner * * * ## If you find this post helpful and are logged into the web interface, please show your appreciation and click on the star below. Thanks. ---
**Author:** ![polezhaevdmi](https://avatars.discourse-cdn.com/v4/letter/p/ccd318/32.png) [@polezhaevdmi](https://forums.suse.com/u/polezhaevdmi)\ **Post date:** [June 15, 2016, 7:15pm UTC](https://forums.suse.com/t/ses-3-installation-issues/28287/6 "2016-06-15T19:15:49Z")
```auto Now open your Web browser and point it to the hostname/IP address of the server where you installed Calamari. ``` Calamari doesn’t respond. [SIZE=1]Service Unavailable The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later. Additionally, a 503 Service Unavailable error was encountered while trying to use an ErrorDocument to handle the request.[/SIZE] Something else should be done prior to access Calamari webpage… ---
**Author:** ![polezhaevdmi](https://avatars.discourse-cdn.com/v4/letter/p/ccd318/32.png) [@polezhaevdmi](https://forums.suse.com/u/polezhaevdmi)\ **Post date:** [June 15, 2016, 7:22pm UTC](https://forums.suse.com/t/ses-3-installation-issues/28287/7 "2016-06-15T19:22:30Z")
Yes, Simon, You are right! Let me laugh for a couple of hours 🙂 ---
**Author:** ![polezhaevdmi](https://avatars.discourse-cdn.com/v4/letter/p/ccd318/32.png) [@polezhaevdmi](https://forums.suse.com/u/polezhaevdmi)\ **Post date:** [June 16, 2016, 11:27am UTC](https://forums.suse.com/t/ses-3-installation-issues/28287/8 "2016-06-16T11:27:50Z")
Investigated a bit more (with an actual manual). - Calamari server and calamari ceph patterns are installed. - Effect doesn’t depend of firewall on or off. - YaST2 firewall management conflicts with SuSEfirewall2, controlled from command line. - YaST2 reports HTTP server is not running, meaning Apache. Is that neccessary to use Calamari ceph administration panel? ---
**Author:** ![rushdir005](https://avatars.discourse-cdn.com/v4/letter/r/858c86/32.png) [@rushdir005](https://forums.suse.com/u/rushdir005)\ **Post date:** [June 29, 2016, 11:58am UTC](https://forums.suse.com/t/ses-3-installation-issues/28287/9 "2016-06-29T11:58:11Z")
I am getting an error when trying to install SES 3.0 on virtualbox. "Internal error:Please report a bug report with logs. Details: Input/Output error @io\_fillbuf fd:28 /mounts/mp\_001/usr/share/YaST2/include/partitioning/custom\_parts\_dialog.rb Caller: /usr/lib64/ruby/2.1.0/rubygems/core\_ext/kernel\_require.rb:55:in ‘require’ But having said that I have managed to install one node. I need two more nodes to setup a SES Cluster. Please help! ---
**Author:** ![polezhaevdmi](https://avatars.discourse-cdn.com/v4/letter/p/ccd318/32.png) [@polezhaevdmi](https://forums.suse.com/u/polezhaevdmi)\ **Post date:** [July 6, 2016, 4:29pm UTC](https://forums.suse.com/t/ses-3-installation-issues/28287/10 "2016-07-06T16:29:41Z")
May be, it will be easier to clone (using GHOST, Acronis or VM image) first installed node and change system settings for new nodes? As for me, I’m installing in VMWare 6 update 1 environment and had no issues with OS or storage extensions installation. ---
**Author:** ![polezhaevdmi](https://avatars.discourse-cdn.com/v4/letter/p/ccd318/32.png) [@polezhaevdmi](https://forums.suse.com/u/polezhaevdmi)\ **Post date:** [July 6, 2016, 5:10pm UTC](https://forums.suse.com/t/ses-3-installation-issues/28287/11 "2016-07-06T17:10:18Z")
OK, the second attempt from scratch. Trying to setup the system with automated deployment via crowbar. Planned the virtual structure as follows. DNS domain: ses.lab.local Nodes: admin (as admin machine, master DNS and NTP gate to i-net) deploy (for crowbar) mon01, mon02, mon03 (secondary DNS and NTP also) osd01, osd02, osd03, osd04 Networks: 172.18.64.0/255.255.252.0 - physical network with access to i-net via proxy 192.168.124.0/255.255.255.0 - virtual network for internal SES communications Each node has 2 virtual NICs: E1000 - to connect to 172.18.64.0 network via virtual switch with link to external LAN with DHCP VXNET3 - to connect to 192.168.124.0 network inside virtual switch only, all IPs are manually assigned The VXNET3 card of the deploy.ses.lab.local node has address 192.168.124.10 to siplify the crowbar setup. Links [https://www.suse.com/documentation/ses-3/book\_storage\_admin/data/ceph\_install\_crowbar.html](https://www.suse.com/documentation/ses-3/book_storage_admin/data/ceph_install_crowbar.html) [https://www.suse.com/documentation/suse-cloud-5/book\_cloud\_deploy/data/sec\_depl\_adm\_inst\_crowbar.html](https://www.suse.com/documentation/suse-cloud-5/book_cloud_deploy/data/sec_depl_adm_inst_crowbar.html) The first note: do NOT install deploy node with text interface only (Base System or something else), as You will waste a lot of time to fix a lot of crowbar initialization problems. I reinstalled the node with graphics and all previous problems were gone. The crowbar installation script (screen install-ses-admin) reports error “crowbar service is not running”. Attempt to start the crowbar service (YAST2 / Services Manager) is unsuccessfull, the status file puma.state is not found. Who forgot to put the file at the right place? 🙂 The installation log is attached. [ATTACH]161[/ATTACH] ---
**Author:** ![polezhaevdmi](https://avatars.discourse-cdn.com/v4/letter/p/ccd318/32.png) [@polezhaevdmi](https://forums.suse.com/u/polezhaevdmi)\ **Post date:** [July 13, 2016, 4:59pm UTC](https://forums.suse.com/t/ses-3-installation-issues/28287/12 "2016-07-13T16:59:19Z")
As the crowbar seems the separate realm of madness, it’s much better to start with ceph-deploy. This will bring furter and faster. OK, Ceph, RADOS Gateway and Calamari operating fine. Now it’s time for iSCSI. As I see, the pools, created with erasure coding use, are unsuitable for block storage image placement. Am I right? If yes, why the erasure code is so discriminized? [CODE]node01:/home/cephadm # ceph osd pool create rbd01 8 8 erasure pool ‘rbd01’ created node01:/home/cephadm # rbd --pool rbd01 create --size=102400 iscsistor01 2016-07-13 16:41:54.110215 7f7308d7fd80 -1 librbd: error adding image to directory: (95) Operation not supported rbd: create error: (95) Operation not supported node01:/home/cephadm # ceph osd pool delete rbd01 rbd01 --yes-i-really-really-mean-it pool ‘rbd01’ removed node01:/home/cephadm # ceph osd pool create rbd01 8 8 pool ‘rbd01’ created node01:/home/cephadm # rbd --pool rbd01 create --size=102400 iscsistor01 node01:/home/cephadm # rbd --pool rbd01 ls iscsistor01[/CODE] ---
**Author:** ![polezhaevdmi](https://avatars.discourse-cdn.com/v4/letter/p/ccd318/32.png) [@polezhaevdmi](https://forums.suse.com/u/polezhaevdmi)\ **Post date:** [July 14, 2016, 5:07pm UTC](https://forums.suse.com/t/ses-3-installation-issues/28287/13 "2016-07-14T17:07:01Z")
Trying to mount CephFS at ‘admin node’ found the kernel client modules for CephFS are not installed and there is no any appropriate package to install from SES3 distribution media. So, both mount -t ceph and mount.ceph commands are unsuccessfull. ‘modprobe ceph’ reports ‘FATAL: Module ceph not found’. The SES3 documentation (15.2.2 “Mounting CephFS”) and DVD should be revised to correspond each other. How to solve that issue? Try to compile client module from [https://github.com/ceph/ceph-client](https://github.com/ceph/ceph-client), or do something else? ---
**Author:** ![polezhaevdmi](https://avatars.discourse-cdn.com/v4/letter/p/ccd318/32.png) [@polezhaevdmi](https://forums.suse.com/u/polezhaevdmi)\ **Post date:** [July 25, 2016, 1:21pm UTC](https://forums.suse.com/t/ses-3-installation-issues/28287/14 "2016-07-25T13:21:19Z")
The SES3 installation media appeared to contain the modules with bug, which is very old. The Calamari tries to connect Oracle at 172.16.79.128/7003 and 172.16.79.128/7002, which makes our CISCO ASA - angry, and network security officer - insane. 🙂 [QUOTE]Jul 25 2016 11:53:57: %ASA-4-106023: Deny tcp src cc-inside:172.18.65.142/58969 dst outside:172.16.79.128/7003 by access-group “CC-INSIDE-IN” [0xe3955a04, 0x0] Jul 25 2016 11:53:57: %ASA-4-106023: Deny tcp src cc-inside:172.18.65.142/56106 dst outside:172.16.79.128/7002 by access-group “CC-INSIDE-IN” [0xe3955a04, 0x0][/QUOTE] The bug report link is [here](http://tracker.ceph.com/issues/10280). It opened 1 year ago and completed in 4 months. As the evaluation registration via proxy was unsuccessfull, thus, the actual solution is: edit the file /usr/lib/python2.7/site-packages/cthulhu/manager/notifier.py and change the 172.18.65.142 addresses with 127.0.0.1, than, restart host with Calamari. ---
**Author:** ![polezhaevdmi](https://avatars.discourse-cdn.com/v4/letter/p/ccd318/32.png) [@polezhaevdmi](https://forums.suse.com/u/polezhaevdmi)\ **Post date:** [July 25, 2016, 6:19pm UTC](https://forums.suse.com/t/ses-3-installation-issues/28287/15 "2016-07-25T18:19:17Z")
Yes, I’m right and the problem is so severe, it doesn’t allow developers to allow a lot of operations with erasure-coded pools: they conflicting with the object nature of Ceph. [http://ceph-users.ceph.narkive.com/eRAVMAu1/why-the-erasure-code-pool-not-support-random-write](http://ceph-users.ceph.narkive.com/eRAVMAu1/why-the-erasure-code-pool-not-support-random-write) Classic storage wins this fight.