PDA

View Full Version : Context Switches Per Second Exceeds



romanperaltasanchez
05-Jan-2014, 16:04
Context Switches Per Second [ Red ]
2737162 meets or exceeds 10000

#==[ Top 10 CPU Processes ]=========================#
%CPU PID USER CMD
3.7 20 root [ksoftirqd/8]
3.2 6 root [ksoftirqd/1]
2.6 8 root [ksoftirqd/2]
2.3 24139 root /bin/bash /sbin/supportconfig
1.9 14 root [ksoftirqd/5]
1.9 10 root [ksoftirqd/3]
1.7 12 root [ksoftirqd/4]
1.6 4 root [ksoftirqd/0]
1.3 22 root [ksoftirqd/9]
1.2 32 root [ksoftirqd/14]
1.2 18 root [ksoftirqd/7]

malcolmlewis
05-Jan-2014, 16:25
On Sun 05 Jan 2014 03:14:02 PM CST, romanperaltasanchez wrote:


Context Switches Per Second [ Red ]
2737162 meets or exceeds 10000

#==[ Top 10 CPU Processes ]=========================#
%CPU PID USER CMD
3.7 20 root [ksoftirqd/8]
3.2 6 root [ksoftirqd/1]
2.6 8 root [ksoftirqd/2]
2.3 24139 root /bin/bash /sbin/supportconfig
1.9 14 root [ksoftirqd/5]
1.9 10 root [ksoftirqd/3]
1.7 12 root [ksoftirqd/4]
1.6 4 root [ksoftirqd/0]
1.3 22 root [ksoftirqd/9]
1.2 32 root [ksoftirqd/14]
1.2 18 root [ksoftirqd/7]




Hi
I'm assuming the post in the SUSE Manager sub forum was incorrect,
since you posted here as well?

If you run vmstat 5, what is the cs value like normally and do you see
it start to increase at a particular time, maybe a database sync?

Also what is the SLE system your seeing this on;


cat /etc/SuSE-release


--
Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
openSUSE 13.1 (Bottle) (x86_64) GNOME 3.10.2 Kernel 3.11.6-4-desktop
If you find this post helpful and are logged into the web interface,
please show your appreciation and click on the star below... Thanks!

romanperaltasanchez
06-Jan-2014, 15:23
Hi Malcolm, thank you for your response.

Please refer to below information as per your request.

#==[ Command ]======================================#
# /bin/uname -a
Linux HQ-iDNS1 2.6.32.12-0.7-default #1 SMP 2010-05-20 11:14:20 +0200 x86_64 x86_64 x86_64 GNU/Linux

#==[ Configuration File ]===========================#
# /etc/SuSE-release
SUSE Linux Enterprise Server 11 (x86_64)
VERSION = 11
PATCHLEVEL = 1


#==[ Configuration File ]===========================#
# /etc/lsb-release
LSB_VERSION="core-2.0-noarch:core-3.2-noarch:core-4.0-noarch:core-2.0-x86_64:core-3.2-x86_64:core-4.0-x86_64"


#==[ Command ]======================================#
# /bin/rpm -qa --queryformat "%{DISTRIBUTION}\n" | sort | uniq
(none)
SLES10 SP2
SUSE Linux Enterprise 11
SUSE Linux Enterprise Server 11

#==[ Verification ]=================================#
# rpm -V SuSEfirewall2-3.6_SVNr208-2.1
# Verification Status: Passed


I do not have access to the DC at the moment hence could not provide the vmstat 5 output. I only have the output from previous logs.

# /usr/bin/vmstat 1 4
procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 0 22593452 87404 537488 0 0 0 1 4 2 0 2 98 0 0
1 0 0 22595540 87404 537476 0 0 0 448 4284 2793062 0 2 98 0 0
1 0 0 22595540 87404 537480 0 0 0 0 3714 2818959 0 2 98 0 0
2 0 0 22595408 87404 537480 0 0 0 0 4061 2800920 0 2 98 0 0

romanperaltasanchez
06-Jan-2014, 15:24
Hi Malcolm, thank you for your response.

Please refer to below information as per your request.

#==[ Configuration File ]===========================#
# /etc/SuSE-release
SUSE Linux Enterprise Server 11 (x86_64)
VERSION = 11
PATCHLEVEL = 1

romanperaltasanchez
06-Jan-2014, 15:25
I do not have access to the DC at the moment hence could not provide the vmstat 5 output. I only have the output from previous logs.

# /usr/bin/vmstat 1 4
procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 0 22593452 87404 537488 0 0 0 1 4 2 0 2 98 0 0
1 0 0 22595540 87404 537476 0 0 0 448 4284 2793062 0 2 98 0 0
1 0 0 22595540 87404 537480 0 0 0 0 3714 2818959 0 2 98 0 0
2 0 0 22595408 87404 537480 0 0 0 0 4061 2800920 0 2 98 0 0

romanperaltasanchez
06-Jan-2014, 15:34
The same issue also happens on other 3 servers. We have observed the server reboot once every 200+ days interval. Is there any way to determine what causing this issue from supportconfig output logs.

Thank you.

malcolmlewis
06-Jan-2014, 16:13
On Mon 06 Jan 2014 02:44:02 PM CST, romanperaltasanchez wrote:


The same issue also happens on other 3 servers. We have observed the
server reboot once every 200+ days interval. Is there any way to
determine what causing this issue from *supportconfig *output logs.

Thank you.




Hi
Interesting, let me ask my SUSE contacts, they may need supportconfig's
from all the systems, let me found out.

--
Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
openSUSE 13.1 (Bottle) (x86_64) GNOME 3.10.2 Kernel 3.11.6-4-desktop
If you find this post helpful and are logged into the web interface,
please show your appreciation and click on the star below... Thanks!

romanperaltasanchez
07-Jan-2014, 16:02
Hi Malcolm, any update on this issue? Let me know where to upload the 'supportconfig' log files.

Appreciate if you could provide some feedback, thank you.

ab
07-Jan-2014, 17:54
You could always drop the supportconfig-created files on
ftp://ftp.novell.com/incoming/ which can only be read by internal folks.
Be sure to post the name of the file when uploading. Some FTP clients are
bugged by the inability to read and may error, but writing there is
definitely possible so if nothing else use the command line 'ftp' client
to do this:

Code:
--------------------
cd /var/log
ftp -a ftp.novell.com
cd incoming
binary
put nts-whatever.bz2
bye
--------------------

--
Good luck.

If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below...

malcolmlewis
08-Jan-2014, 01:37
On Tue 07 Jan 2014 03:04:02 PM CST, romanperaltasanchez wrote:


Hi Malcolm, any update on this issue? Let me know where to upload the
'supportconfig' log files.

Appreciate if you could provide some feedback, thank you.




Hi
A couple of things to consider, upgrading to the latest supported
release, now at SLE 11 SP3, else do you have LTSS for the servers in
question, without that, then getting some SUSE support will be an issue.

--
Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
openSUSE 13.1 (Bottle) (x86_64) GNOME 3.10.2 Kernel 3.11.6-4-desktop
If you find this post helpful and are logged into the web interface,
please show your appreciation and click on the star below... Thanks!

romanperaltasanchez
08-Jan-2014, 11:08
Hi Malcolm,

Please refer to below result of vmstat for your reference. Let me know your findings, thank you.

HQ-iDNS@HQ-iDNS1:~> uptime
1:44pm up 11 days 5:39, 1 user, load average: 0.00, 0.02, 0.04

HQ-iDNS@HQ-iDNS1:~> vmstat 5 20
procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 0 21452088 97292 1642608 0 0 0 1 3 1 0 2 98 0 0
1 0 0 21451856 97292 1642616 0 0 0 33 4148 1660628 0 2 98 0 0
1 0 0 21451280 97292 1642612 0 0 0 13 3937 1794703 0 2 98 0 0
1 0 0 21452088 97292 1642612 0 0 0 53 3975 2282438 0 2 98 0 0
0 0 0 21451964 97292 1642612 0 0 0 31 3974 2525368 0 2 98 0 0
1 0 0 21452304 97292 1642612 0 0 0 2 3985 2555776 0 2 98 0 0
0 0 0 21452484 97292 1642612 0 0 0 6 4206 2544244 0 2 98 0 0
1 0 0 21452600 97292 1642644 0 0 0 34 4018 2542588 0 2 98 0 0
0 0 0 21453216 97292 1642644 0 0 0 8 4091 2556794 0 2 98 0 0
0 0 0 21453264 97292 1642644 0 0 0 25 4174 1680464 0 2 98 0 0
0 0 0 21454156 97292 1642624 0 0 0 30 4043 2083160 0 2 98 0 0
0 0 0 21453908 97292 1642624 0 0 0 0 4144 2532364 0 2 98 0 0
1 0 0 21454012 97292 1642624 0 0 0 9 3810 2544554 0 3 97 0 0
1 0 0 21454064 97292 1642620 0 0 0 12 4302 2414714 0 2 97 0 0
0 0 0 21454404 97292 1642620 0 0 0 9 3713 2658080 0 2 98 0 0
1 0 0 21453860 97292 1642616 0 0 0 8 3889 2688259 0 2 98 0 0
2 0 0 21454216 97292 1642612 0 0 0 15 3767 2662618 0 2 98 0 0
1 0 0 21453668 97292 1642612 0 0 0 9 3990 2547906 0 2 98 0 0
0 0 0 21453168 97292 1642616 0 0 0 13 3761 2626326 0 2 98 0 0
0 0 0 21452488 97292 1642616 0 0 0 11 3972 2652037 0 2 98 0 0

HQ-eDNS@HQ-eDNS1:~> uptime
1:43pm up 90 days 0:11, 1 user, load average: 0.89, 0.83, 0.82
HQ-eDNS@HQ-eDNS1:~> vmstat 5 20
procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
1 0 0 20196756 146968 2356428 0 0 0 1 0 0 4 2 94 0 0
2 0 0 20196908 146968 2356420 0 0 0 6 3931 1725347 6 2 92 0 0
2 0 0 20196748 146968 2356412 0 0 0 9 4367 2626402 6 2 91 0 0
1 0 0 20197604 146968 2356408 0 0 0 34 4248 2689039 6 2 91 0 0
2 0 0 20198228 146968 2356416 0 0 0 6 4830 2692594 6 2 91 0 0
1 0 0 20199184 146968 2356424 0 0 0 6 4095 2734745 6 2 91 0 0
2 0 0 20197120 146968 2356424 0 0 0 6 3724 2699805 6 2 91 0 0
1 0 0 20196832 146968 2356424 0 0 0 11 4232 1515543 6 1 92 0 0
2 0 0 20193016 146968 2356452 0 0 0 16 4343 939612 7 1 92 0 0
2 0 0 20193048 146968 2356412 0 0 0 19 4326 2824990 6 2 91 0 0
1 0 0 20193028 146968 2356432 0 0 0 20 4641 2814314 6 2 91 0 0
2 0 0 20193872 146968 2356440 0 0 0 6 4589 2743772 6 2 91 0 0
1 0 0 20194872 146968 2356440 0 0 0 8 4434 2674108 6 2 92 0 0
2 0 0 20194964 146968 2356432 0 0 0 13 4906 2724691 6 2 91 0 0
2 0 0 20195704 146968 2356412 0 0 0 10 3973 982867 6 1 93 0 0
1 0 0 20197048 146968 2356412 0 0 0 7 4572 2272820 6 2 92 0 0
1 0 0 20196900 146968 2356436 0 0 0 8 4262 319001 6 1 93 0 0
1 0 0 20197396 146968 2356436 0 0 0 16 4176 20163 6 0 93 0 0
3 0 0 20198456 146968 2356432 0 0 0 0 4032 163078 6 0 93 0 0
4 0 0 20193320 146968 2356448 0 0 0 4 4399 1886195 7 2 92 0 0


TEL-iDNS@TEL-iDNS2:~> uptime
1:44pm up 47 days 11:50, 1 user, load average: 0.08, 0.02, 0.01
TEL-iDNS@TEL-iDNS2:~>
TEL-iDNS@TEL-iDNS2:~>
TEL-iDNS@TEL-iDNS2:~> vmstat 5 20
procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
1 0 0 22596936 91572 847288 0 0 0 0 0 0 0 2 98 0 0
2 0 0 22596824 91572 847288 0 0 0 7 4344 2662016 0 2 98 0 0
1 0 0 22597624 91572 847288 0 0 0 13 4255 2720991 0 2 98 0 0
1 0 0 22594768 91572 847316 0 0 0 45 4165 2605152 0 2 98 0 0
1 0 0 22595668 91572 847328 0 0 0 14 3949 2794644 0 2 98 0 0
1 0 0 22596212 91572 847324 0 0 0 0 4512 2690459 0 2 98 0 0
1 0 0 22597144 91572 847312 0 0 0 14 4429 2745051 0 2 98 0 0
1 0 0 22596196 91572 847292 0 0 0 11 4090 2708583 0 2 98 0 0
2 0 0 22597048 91572 847288 0 0 0 9 4123 2766559 0 2 98 0 0
0 0 0 22596800 91572 847288 0 0 0 9 4112 2691188 0 2 98 0 0
1 0 0 22596976 91572 847292 0 0 0 19 3942 2766484 0 2 98 0 0
2 0 0 22596920 91572 847304 0 0 0 0 4449 2749358 0 2 98 0 0
0 0 0 22597080 91572 847300 0 0 0 3 4154 2708302 0 2 98 0 0
0 0 0 22597060 91572 847296 0 0 0 7 4435 2676674 0 2 98 0 0
3 0 0 22597504 91572 847292 0 0 0 16 4621 2749607 0 2 98 0 0
0 0 0 22597048 91572 847296 0 0 0 10 4192 2761957 0 2 98 0 0
1 0 0 22597156 91572 847296 0 0 0 10 4650 2653542 0 2 98 0 0
1 0 0 22596440 91572 847296 0 0 0 5 4158 2648237 0 2 98 0 0
1 0 0 22597028 91572 847292 0 0 0 3 3846 2635196 0 2 98 0 0
0 0 0 22596500 91572 847296 0 0 0 11 3815 2646626 0 2 98 0 0


TEL-eDNS@TEL-eDNS2:~> uptime
1:46pm up 51 days 1:23, 1 user, load average: 0.32, 0.33, 0.19

TEL-eDNS@TEL-eDNS2:~> vmstat 5 20
procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
1 0 0 20757808 142004 2577988 0 0 0 0 0 0 0 2 98 0 0
1 0 0 20758256 142004 2577992 0 0 0 7 4421 2773257 0 2 98 0 0
2 0 0 20759328 142004 2577992 0 0 0 10 4126 2737564 0 2 98 0 0
0 0 0 20759148 142004 2577984 0 0 0 1 4127 2713943 0 2 98 0 0
0 0 0 20759180 142004 2577984 0 0 0 8 3889 2795331 0 2 98 0 0
1 0 0 20758868 142004 2577984 0 0 0 14 4317 2611316 0 2 98 0 0
1 0 0 20759008 142004 2577984 0 0 0 8 4044 2772481 0 2 98 0 0
0 0 0 20758180 142004 2577980 0 0 0 3 4204 2698640 0 2 98 0 0
2 0 0 20759132 142004 2577980 0 0 0 4 4307 2528843 0 2 98 0 0
0 0 0 20758352 142004 2577980 0 0 0 3 4385 2184150 0 2 98 0 0
1 0 0 20758968 142004 2577980 0 0 0 7 4252 2105741 0 2 98 0 0
0 0 0 20758992 142004 2577976 0 0 0 3 4337 2221985 0 2 98 0 0
1 0 0 20759644 142004 2577976 0 0 0 8 4023 2617581 0 2 98 0 0
2 0 0 20759252 142004 2577976 0 0 0 6 3917 2514349 0 2 98 0 0
1 0 0 20759584 142004 2577976 0 0 0 3 3750 2006943 0 2 98 0 0
3 0 0 20759812 142004 2577976 0 0 0 0 4033 1990970 0 1 99 0 0
1 0 0 20759508 142004 2577980 0 0 0 11 3910 2769837 0 2 98 0 0
1 0 0 20759792 142004 2577980 0 0 0 10 3753 2758156 0 2 98 0 0
1 0 0 20758752 142004 2577980 0 0 0 3 4296 2616506 0 2 98 0 0
1 0 0 20758992 142004 2577980 0 0 0 9 4342 2571069 0 2 98 0 0

romanperaltasanchez
17-Jan-2014, 01:28
Hi Malcolm,

Would you be able to share the procedure of upgrading from SLES 11 SP1 to SLES 11 SP3. Do we need to re-install all the application during the upgrade?

Thank you.

malcolmlewis
17-Jan-2014, 02:46
On Fri 17 Jan 2014 12:34:02 AM CST, romanperaltasanchez wrote:


Hi Malcolm,

Would you be able to share the procedure of upgrading from SLES 11 SP1
to SLES 11 SP3. Do we need to re-install all the application during the
upgrade?

Thank you.




Hi
Here are the release notes;
https://www.suse.com/releasenotes/x86_64/SUSE-SLES/11-SP3/#

Specifically this section;
https://www.suse.com/releasenotes/x86_64/SUSE-SLES/11-SP3/#Update.SP1ToSP3
and
https://www.suse.com/releasenotes/x86_64/SUSE-SLES/11-SP3/#Update.General.Sequenced

So you would need to upgrade to SP2, then onto SP3.

I'm not sure about your application, are these virtual machines? If so,
then create a snapshot and run through an upgrade as a test on the
snapshot vm.

--
Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
openSUSE 13.1 (Bottle) (x86_64) GNOME 3.10.2 Kernel 3.11.6-4-desktop
If you find this post helpful and are logged into the web interface,
please show your appreciation and click on the star below... Thanks!

KBOYLE
17-Jan-2014, 04:20
romanperaltasanchez wrote:

> Would you be able to share the procedure of upgrading from SLES 11 SP1
> to SLES 11 SP3.

Here's another document with pretty much everything you need to know:
TID 7012368 How to upgrade to SLES/SLED 11 SP3
https://www.suse.com/support/kb/doc.php?id=7012368

--
Kevin Boyle - Knowledge Partner
If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below...