# Rancher 2.x Rook.io problem **URL:** \ **Category:** SUSE Rancher Prime\ **Created:** [May 22, 2018, 9:32am UTC](https://forums.suse.com/t/rancher-2-x-rook-io-problem/10503 "2018-05-22T09:32:01Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Victor\_H\_Saez](https://avatars.discourse-cdn.com/v4/letter/v/d26b3c/32.png) [@Victor\_H\_Saez](https://forums.suse.com/u/Victor_H_Saez)\ **Post date:** [May 22, 2018, 9:32am UTC](https://forums.suse.com/t/rancher-2-x-rook-io-problem/10503/1 "2018-05-22T09:32:02Z")
Hi, I have a rancher 2.x managed cluster deployed over digitaloceal. It has three nodes based on ubuntu 16.04. I have followed this guide [https://rook.io/docs/rook/master/quickstart.html](https://rook.io/docs/rook/master/quickstart.html) It seems everythin is OK. The PVC are bound but the pods fail to deploy. All rook related pod are running. This is the output of get pvc wp-pv-claim Bound pvc-0875428f-5d0a-11e8-9223-0ab22348a3dd 1Gi RWO rook-block 18h This is the output of the pod description Warning FailedMount 34s (x479 over 18h) kubelet, node3 Unable to mount volumes for pod “wordpress-mysql-5bbddff49c-xr786\_default(00275a1b-5d0a-11e8-9223-0ab22348a3dd)”: timeout expired waiting for volumes to attach or mount for pod “default”/“wordpress-mysql-5bbddff49c-xr786”. list of unmounted volumes=[mysql-persistent-storage]. list of unattached volumes=[mysql-persistent-storage default-token-bqjlr] ---
**Author:** ![majestic](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/majestic/32/4074_2.png) [@majestic](https://forums.suse.com/u/majestic)\ **Post date:** [May 28, 2018, 1:22pm UTC](https://forums.suse.com/t/rancher-2-x-rook-io-problem/10503/2 "2018-05-28T13:22:55Z")
Hi there, I have the exact same problem here, have you managed to get any further with this? Regards, ---
**Author:** ![Victor\_H\_Saez](https://avatars.discourse-cdn.com/v4/letter/v/d26b3c/32.png) [@Victor\_H\_Saez](https://forums.suse.com/u/Victor_H_Saez)\ **Post date:** [May 28, 2018, 2:51pm UTC](https://forums.suse.com/t/rancher-2-x-rook-io-problem/10503/3 "2018-05-28T14:51:59Z")
Hi, I think I found the solution for my problem. First, I access everynode and make /var/lib/kubelet/volumeplugins chmod 777. Then on the rook-operator.yaml I uncommented the two lines regarding the volumeplugin folder location. This has worked for me on RancherOS and Ubuntu 16.04 nodes I hope you can solve your problem also ---
**Author:** ![majestic](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/majestic/32/4074_2.png) [@majestic](https://forums.suse.com/u/majestic)\ **Post date:** [May 31, 2018, 3:56am UTC](https://forums.suse.com/t/rancher-2-x-rook-io-problem/10503/4 "2018-05-31T03:56:56Z")
Thanks very much, will check it out. I did get this working on centos the other day just by adding the two path bits. An oversight I missed originally. Ideally you never want to use 777 and I’m 99% sure it should not be needed. I haven’t yet retested this with rancher, I switched to kubespey via ansible for now due to some other rancher bugs which was annoying me no end. ---
**Author:** ![Negash](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/negash/32/675_2.png) [@Negash](https://forums.suse.com/u/Negash)\ **Post date:** [June 6, 2018, 8:11am UTC](https://forums.suse.com/t/rancher-2-x-rook-io-problem/10503/5 "2018-06-06T08:11:38Z")
I have 3 nodes with RancherOS 1.3.0 (and testing with 3 nodes ros 1.4.0) ```auto mounts: - - /dev/sdb - /data - ext4 - "" rancher: autologin: ttyS0 console: ubuntu docker: engine: docker-17.03.2-ce graph: /data storage_driver: overlay2 ``` with the docker installed in the `/data` directory. set installed rook with `/var/lib/kubelet/volumeplugins` (data on the host appeared) ```auto rancher@docker-vm13:~$ ls /var/lib/kubelet/volumeplugins ceph.rook.io~rook ceph.rook.io~rook-ceph-system rook.io~rook rook.io~rook-ceph-system ``` create rook with mysql/wordpress example ```auto kubectl create -f operator.yaml kubectl create -f cluster.yaml kubectl create -f https://raw.githubusercontent.com/rook/rook/master/cluster/examples/kubernetes/mysql.yaml kubectl create -f https://raw.githubusercontent.com/rook/rook/master/cluster/examples/kubernetes/wordpress.yaml ``` creates pvm ![Screenshot%20(1)](https://us1.discourse-cdn.com/flex022/uploads/suse/original/2X/3/3dfc3b0a7b84c9dd887d4508b14c8d77cf9799c1.png) but the containers do not start ![Screenshot](https://us1.discourse-cdn.com/flex022/uploads/suse/original/2X/9/9d73a5f068c704ef52b81d6b2d62638e473a0b1a.png) kubectl -n rook-ceph-system logs -f rook-discover-5lzkv (infinity write this every 30 seconds) ```auto 2018-06-06 08:05:53.282729 I | rook-discover: updating device configmap 2018-06-06 08:05:53.282796 I | exec: Running command: lsblk --all --noheadings --list --output KNAME 2018-06-06 08:05:53.288272 I | exec: Running command: lsblk /dev/loop1 --bytes --nodeps --pairs --output SIZE,ROTA,RO,TYPE,PKNAME 2018-06-06 08:05:53.291832 W | inventory: skipping device loop1: Failed to complete 'lsblk /dev/loop1': exit status 1. 2018-06-06 08:05:53.291865 I | exec: Running command: lsblk /dev/sdb --bytes --nodeps --pairs --output SIZE,ROTA,RO,TYPE,PKNAME 2018-06-06 08:05:53.294755 I | exec: Running command: udevadm info --query=property /dev/sdb 2018-06-06 08:05:53.297263 I | exec: Running command: udevadm info --query=property /dev/sdb 2018-06-06 08:05:53.299891 I | exec: Running command: lsblk /dev/loop6 --bytes --nodeps --pairs --output SIZE,ROTA,RO,TYPE,PKNAME 2018-06-06 08:05:53.302668 W | inventory: skipping device loop6: Failed to complete 'lsblk /dev/loop6': exit status 1. 2018-06-06 08:05:53.302707 I | exec: Running command: lsblk /dev/loop4 --bytes --nodeps --pairs --output SIZE,ROTA,RO,TYPE,PKNAME 2018-06-06 08:05:53.305731 W | inventory: skipping device loop4: Failed to complete 'lsblk /dev/loop4': exit status 1. 2018-06-06 08:05:53.305762 I | exec: Running command: lsblk /dev/sr0 --bytes --nodeps --pairs --output SIZE,ROTA,RO,TYPE,PKNAME 2018-06-06 08:05:53.308949 I | exec: Running command: lsblk /dev/loop2 --bytes --nodeps --pairs --output SIZE,ROTA,RO,TYPE,PKNAME 2018-06-06 08:05:53.311852 W | inventory: skipping device loop2: Failed to complete 'lsblk /dev/loop2': exit status 1. 2018-06-06 08:05:53.311896 I | exec: Running command: lsblk /dev/loop0 --bytes --nodeps --pairs --output SIZE,ROTA,RO,TYPE,PKNAME 2018-06-06 08:05:53.314707 W | inventory: skipping device loop0: Failed to complete 'lsblk /dev/loop0': exit status 1. 2018-06-06 08:05:53.314748 I | exec: Running command: lsblk /dev/sda --bytes --nodeps --pairs --output SIZE,ROTA,RO,TYPE,PKNAME 2018-06-06 08:05:53.318033 I | exec: Running command: udevadm info --query=property /dev/sda 2018-06-06 08:05:53.321099 I | exec: Running command: udevadm info --query=property /dev/sda 2018-06-06 08:05:53.324030 I | exec: Running command: lsblk /dev/sda1 --bytes --nodeps --pairs --output SIZE,ROTA,RO,TYPE,PKNAME 2018-06-06 08:05:53.328649 I | exec: Running command: udevadm info --query=property /dev/sda1 2018-06-06 08:05:53.331948 I | exec: Running command: lsblk /dev/loop7 --bytes --nodeps --pairs --output SIZE,ROTA,RO,TYPE,PKNAME 2018-06-06 08:05:53.335981 W | inventory: skipping device loop7: Failed to complete 'lsblk /dev/loop7': exit status 1. 2018-06-06 08:05:53.336022 I | exec: Running command: lsblk /dev/loop5 --bytes --nodeps --pairs --output SIZE,ROTA,RO,TYPE,PKNAME 2018-06-06 08:05:53.338982 W | inventory: skipping device loop5: Failed to complete 'lsblk /dev/loop5': exit status 1. 2018-06-06 08:05:53.339013 I | exec: Running command: lsblk /dev/loop3 --bytes --nodeps --pairs --output SIZE,ROTA,RO,TYPE,PKNAME 2018-06-06 08:05:53.341732 W | inventory: skipping device loop3: Failed to complete 'lsblk /dev/loop3': exit status 1. 2018-06-06 08:05:53.341768 I | exec: Running command: lsblk /dev/sdb --bytes --pairs --output NAME,SIZE,TYPE,PKNAME 2018-06-06 08:05:53.344809 I | exec: Running command: udevadm info --query=property /dev/sdb 2018-06-06 08:05:53.347293 I | exec: Running command: lsblk /dev/sda --bytes --pairs --output NAME,SIZE,TYPE,PKNAME 2018-06-06 08:05:53.350987 I | exec: Running command: udevadm info --query=property /dev/sda1 2018-06-06 08:05:53.354279 I | exec: Running command: udevadm info --query=property /dev/sda1 2018-06-06 08:05:53.357668 I | exec: Running command: udevadm info --query=property /dev/sda 2018-06-06 08:05:53.360444 I | rook-discover: available devices: [{Name:sdb Parent: HasChildren:false DevLinks:/dev/disk/by-id/ata-QEMU_HARDDISK_QM00007 /dev/disk/by-uuid/a659a83d-4269-4816-92f5-f118da828da6 Size:107374182400 UUID:a659a83d-4269-4816-92f5-f118da828da6 Serial:QEMU_HARDDISK_QM00007 Type:disk Rotational:true Readonly:false Partitions:[] Filesystem:ext4 Vendor: Model:QEMU_HARDDISK WWN: WWNVendorExtension: Empty:true} {Name:sda Parent: HasChildren:false DevLinks:/dev/disk/by-id/ata-QEMU_HARDDISK_QM00005 Size:10737418240 UUID:4003f5e1 Serial:QEMU_HARDDISK_QM00005 Type:disk Rotational:true Readonly:false Partitions:[{Name:sda1 Size:10735321088 Label: Filesystem:ext4}] Filesystem: Vendor: Model:QEMU_HARDDISK WWN: WWNVendorExtension: Empty:true}] ``` ---
**Author:** ![neo1](https://avatars.discourse-cdn.com/v4/letter/n/b9bd4f/32.png) [@neo1](https://forums.suse.com/u/neo1)\ **Post date:** [November 28, 2018, 1:17pm UTC](https://forums.suse.com/t/rancher-2-x-rook-io-problem/10503/6 "2018-11-28T13:17:48Z")
I tried the chmod 777 but it did not work. Any other solutions to this one? ---
**Author:** ![Erik\_Van\_Norstrand](https://avatars.discourse-cdn.com/v4/letter/e/ee59a6/32.png) [@Erik\_Van\_Norstrand](https://forums.suse.com/u/Erik_Van_Norstrand)\ **Post date:** [December 5, 2018, 3:28pm UTC](https://forums.suse.com/t/rancher-2-x-rook-io-problem/10503/8 "2018-12-05T15:28:17Z")
@neo1 - You do NOT need to chmod 777 `/var/lib/kubelet/volumeplugins` to make rook work. You need to configure the operator to use that folder for it’s flexvolume as documented here: [https://rook.io/docs/rook/v0.8/flexvolume.html](https://rook.io/docs/rook/v0.8/flexvolume.html) @Victor_H_Saez - The reason your getting that error is due to the OSD’s not properly being created. Check your cluster config and operator logs. ---
**Author:** ![neo1](https://avatars.discourse-cdn.com/v4/letter/n/b9bd4f/32.png) [@neo1](https://forums.suse.com/u/neo1)\ **Post date:** [December 5, 2018, 6:36pm UTC](https://forums.suse.com/t/rancher-2-x-rook-io-problem/10503/9 "2018-12-05T18:36:35Z")
@Erik_Van_Norstrand Thanks! that solved the problem!