[Emc-users] latency excursions

Discussion:

dave

2017-07-23 15:48:19 UTC

***@linuxcnc:~$ uname -a
Linux linuxcnc 3.4-9-rtai-686-pae #1 SMP PREEMPT Debian 3.4.55-4linuxcnc
i686 i686 i386 GNU/Linux

task: 103857488 cycles, min=0.000049, max=0.474214, avg=0.010993, 86
latency excursions (> 10x expected cycle time of 0.010000s)

task: 25439189 cycles, min=0.000049, max=0.205898, avg=0.010989, 17
latency excursions (> 10x expected cycle time of 0.010000s)

I've done nothing to tune this cpu and as far as I can tell the motion
is ok.

Still it leave me uncomfortable.

What do I need to change to clean this uo?

TIA

Dave

Nicklas Karlsson

2017-07-23 17:01:35 UTC

Permalink

For me this or something similar i happening one of my computers at every startup. I think it is running smootly in between now but this have not always been the case, I removed a graphics card a while ago and might have changed things or simply a newer kernel.

On Sun, 23 Jul 2017 08:48:19 -0700

Post by dave
Linux linuxcnc 3.4-9-rtai-686-pae #1 SMP PREEMPT Debian 3.4.55-4linuxcnc
i686 i686 i386 GNU/Linux
task: 103857488 cycles, min=0.000049, max=0.474214, avg=0.010993, 86
latency excursions (> 10x expected cycle time of 0.010000s)
task: 25439189 cycles, min=0.000049, max=0.205898, avg=0.010989, 17
latency excursions (> 10x expected cycle time of 0.010000s)
I've done nothing to tune this cpu and as far as I can tell the motion
is ok.
Still it leave me uncomfortable.
What do I need to change to clean this uo?
TIA
Dave
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Emc-users mailing list
https://lists.sourceforge.net/lists/listinfo/emc-users

Peter C. Wallace

2017-07-23 17:14:44 UTC

Permalink

Date: Sun, 23 Jul 2017 08:48:19 -0700
Reply-To: "Enhanced Machine Controller (EMC)"
Subject: [Emc-users] latency excursions
Linux linuxcnc 3.4-9-rtai-686-pae #1 SMP PREEMPT Debian 3.4.55-4linuxcnc i686
i686 i386 GNU/Linux
task: 103857488 cycles, min=0.000049, max=0.474214, avg=0.010993, 86 latency
excursions (> 10x expected cycle time of 0.010000s)
task: 25439189 cycles, min=0.000049, max=0.205898, avg=0.010989, 17 latency
excursions (> 10x expected cycle time of 0.010000s)
I've done nothing to tune this cpu and as far as I can tell the motion is ok.
Still it leave me uncomfortable.
What do I need to change to clean this uo?
TIA
Dave

These are not real-time latency issues but rather delays in task
so are not a actual problem unless they are so long that they cause
a disruption the the non-real time tasks linuxCNC is doing (user interface
etc)

These delays will not be reduced by any real time latency tuning of the system
( and may well be made worse by things that slow the userland performance like
isocpus )

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Emc-users mailing list
https://lists.sourceforge.net/lists/listinfo/emc-users

Peter Wallace
Mesa Electronics

(\__/)
(='.'=) This is Bunny. Copy and paste bunny into your
(")_(") signature to help him gain world domination.

dave

2017-07-23 17:37:09 UTC

Permalink

Post by Peter C. Wallace

Date: Sun, 23 Jul 2017 08:48:19 -0700
Reply-To: "Enhanced Machine Controller (EMC)"
To: "Enhanced Machine Controller (EMC)"
Subject: [Emc-users] latency excursions
Linux linuxcnc 3.4-9-rtai-686-pae #1 SMP PREEMPT Debian
3.4.55-4linuxcnc i686 i686 i386 GNU/Linux
task: 103857488 cycles, min=0.000049, max=0.474214, avg=0.010993, 86
latency excursions (> 10x expected cycle time of 0.010000s)
task: 25439189 cycles, min=0.000049, max=0.205898, avg=0.010989, 17
latency excursions (> 10x expected cycle time of 0.010000s)
I've done nothing to tune this cpu and as far as I can tell the motion is ok.
Still it leave me uncomfortable.
What do I need to change to clean this uo?
TIA
Dave

These are not real-time latency issues but rather delays in task
so are not a actual problem unless they are so long that they cause
a disruption the the non-real time tasks linuxCNC is doing (user
interface etc)
These delays will not be reduced by any real time latency tuning of
the system ( and may well be made worse by things that slow the
userland performance like isocpus )
Peter Wallace
Mesa Electronics
(\__/)
(='.'=) This is Bunny. Copy and paste bunny into your
(")_(") signature to help him gain world domination.
Thanks Peter,

I suppose I should have picked up on the 'task'. I'm going to use age as
an excuse. ;-)

I re-tuned the x axis yesterday for some improvement; following error is
somewhere around 2E-5.

Dave

Post by Peter C. Wallace
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Emc-users mailing list
https://lists.sourceforge.net/lists/listinfo/emc-users

Peter C. Wallace

2017-07-23 18:14:21 UTC

Permalink

Date: Sun, 23 Jul 2017 10:37:09 -0700
Reply-To: "Enhanced Machine Controller (EMC)"
Subject: Re: [Emc-users] latency excursions

Post by Peter C. Wallace

Date: Sun, 23 Jul 2017 08:48:19 -0700
Reply-To: "Enhanced Machine Controller (EMC)"
Subject: [Emc-users] latency excursions
Linux linuxcnc 3.4-9-rtai-686-pae #1 SMP PREEMPT Debian 3.4.55-4linuxcnc
i686 i686 i386 GNU/Linux
task: 103857488 cycles, min=0.000049, max=0.474214, avg=0.010993, 86
latency excursions (> 10x expected cycle time of 0.010000s)
task: 25439189 cycles, min=0.000049, max=0.205898, avg=0.010989, 17
latency excursions (> 10x expected cycle time of 0.010000s)
I've done nothing to tune this cpu and as far as I can tell the motion is ok.
Still it leave me uncomfortable.
What do I need to change to clean this uo?
TIA
Dave

These are not real-time latency issues but rather delays in task
so are not a actual problem unless they are so long that they cause
a disruption the the non-real time tasks linuxCNC is doing (user interface
etc)
These delays will not be reduced by any real time latency tuning of the
system ( and may well be made worse by things that slow the userland
performance like isocpus )
Peter Wallace
Mesa Electronics
(\__/)
(='.'=) This is Bunny. Copy and paste bunny into your
(")_(") signature to help him gain world domination.
Thanks Peter,

I suppose I should have picked up on the 'task'. I'm going to use age as an
excuse. ;-)
I re-tuned the x axis yesterday for some improvement; following error is
somewhere around 2E-5.
Dave

You can do a little experiment with delaying task by starting LinuxCNC from
the command line and then suspending it while running gcode with control Z.
This does not disrupt motion in progress, as long as the delay is not too
long, and if it is too long you will get:

USRMOT: ERROR: command timeout

(after the control Z, you can resume task/user-interface with "fg 1")

Peter Wallace
Mesa Electronics

(\__/)
(='.'=) This is Bunny. Copy and paste bunny into your
(")_(") signature to help him gain world domination.

Gene Heskett

2017-07-23 17:18:52 UTC

Permalink

How many cores does that cpu have? And if its an intel, is hyperthreading
turned on? Thats a bit hard on the latencytest results. Turn it off in
the bios.

If 2 or more cores, it may help to add to the grub line that loads the
kernel, isolcpus=lastcore, where lastcore could be 2, 4, or even 8 if
you've a really new barn burner machine.

With that kernel, your dmesg should contain a line:

Brought up 2 CPUs

The 2 will vary depending on the cpu in that machine, so in this case the
kernel loading line in /boot/grub/grub.cfg might have "isolcpus=1"
appended to it, without the dbl-quotes.

This isolates that core from the scheduler, linuxcnc sees it and uses
that core for the realtime part, having it all to itself. But a monitor
such as top or htop, can't tell you how hard that cpu core is running.

I have one machine still doing software stepping, and that line in its
grub has "isolcpus=1 idle=poll", which seemed to help the latency there.

And just for my own enjoyment, I usually remove the splash and quiet
options. Splash covers up any boot msgs with an ugly graphic, and quiet
turns the boot msgs off, which makes the boot faster when it doesn't
have to do all the screen stuffs. That way I can scroll back and check
that everything looks ok. Or, if mount decides to fsck the drive, you at
least have something on screen to explain the delayed boot.

I don't normally care how long it takes to boot, only that it boots
normally, doing what it has to do to make it run as usual for that
machine.

I HTH Dave.

Cheers, Gene Heskett

--
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>