You are not registered yet. Please click here to register!


 
 
plc storereviewsdownloads
This board is for PLC Related Q&A ONLY. Please DON'T use it for advertising, etc.
 
Try our online PLC Simulator- FREE.  Click here now to try it.

New Here? Please read this important info!!!


Go Back   PLCS.net - Interactive Q & A > PLCS.net - Interactive Q & A > LIVE PLC Questions And Answers

Reply
 
Thread Tools Display Modes
Old March 4th, 2022, 02:17 PM   #1
id10t_error
Lifetime Supporting Member
United States

id10t_error is offline
 
id10t_error's Avatar
 
Join Date: Oct 2017
Location: California
Posts: 92
High Energy particles from the cosmos causing bit flips

After watching The Universe is Hostile to Computers from Veritasium, I suddenly remembered a lost weekend years ago trying to reproduce the same unexpected behavior. So, operators 1,200,431, particle 1, if anyone is keeping score.
__________________
"We make the solutions up out of modules, so they're all standard, ... They're built for security, reliability, ease of expansion and ease of changing. They're the ideas we had many years ago that tend to get lost today." -Ken Olsen co-founded Digital Equipment Corporation (DEC)
  Reply With Quote
Old March 4th, 2022, 03:44 PM   #2
Nova5
Member
United States

Nova5 is offline
 
Join Date: May 2013
Location: Northern Snowball
Posts: 563
Hrm... More... PEBKAC.
__________________
Problems that go away on their own mystically are not solved. They will return.
  Reply With Quote
Old March 4th, 2022, 04:52 PM   #3
Peter Nachtwey
Member
United States

Peter Nachtwey is offline
 
Peter Nachtwey's Avatar
 
Join Date: Apr 2002
Location: The land of the woke and the home of the broke
Posts: 7,966
That is why our newer controllers use ECC.
__________________
"Living is easy with eyes closed, misunderstanding all you see...." Strawberry Fields Forever, John Lennon
  Reply With Quote
Old March 5th, 2022, 07:15 AM   #4
ndzied1
Lifetime Supporting Member
United States

ndzied1 is online now
 
ndzied1's Avatar
 
Join Date: Aug 2002
Location: Chicago, Illinois
Posts: 2,769
This reminds me of story an engineer told me when I was just starting out. He was working to analyze the heating of train cars and was doing testing out in the field.

They had a train car decked out with very accurate temperature sensors. They started noticing on clear nights some of the readings were slightly below the ambient temperature. After multiple checks the sensors were deemed correct. After a little more head scratching and calculating the found out that on clear nights there was some radiation heat transfer to space that they had not considered. The car was warmer than space so the direction of heat transfer was out of the car which made it cool. It was a very small amount but measurable in this case with their accurate instruments.
__________________
nOrM
======================
nOrM=Norman Dziedzic Jr.
I've never been to China but my phone has.

Last edited by ndzied1; March 5th, 2022 at 07:16 AM. Reason: Spelling
  Reply With Quote
Old March 5th, 2022, 10:55 AM   #5
I_Automation
Member
United States

I_Automation is offline
 
I_Automation's Avatar
 
Join Date: Jun 2020
Location: Detroit, Michigan USA
Posts: 1,135
The first time I heard of this was an election recount.


In a small town one lady overwhelmingly won, but the odd part was she won by a lot more than the number of all the registered voters that could have voted.


Turns out the recount was off by precisely 0010000000000000 binary, somehow in the original count bit #14 got turned on and it couldn't be replicated. There weren't 8,192 registered voters in that precinct.



If bit #27 got tripped that would give someone running for president an extra 67.1 million votes
__________________
ivanovaautomation.com


Some Jobs Are Best Started With A Sawzall And A Dumpster On Castors
  Reply With Quote
Old March 5th, 2022, 12:02 PM   #6
Dayvieboy
Lifetime Supporting Member
United States

Dayvieboy is offline
 
Dayvieboy's Avatar
 
Join Date: Jun 2013
Location: San Francisco Bay Area
Posts: 108
Bit Flipping Randomly

Had a project @ a Blommer Chocolate where I have consulted for 30 years now

In a paste mixing system with 6 PLC's, 10 pumps & ~300 valves
Bits would get flipped that operators had no access to from a GUI

Emergency service call, production down.
Pumps & Valves world go to disabled or test states in the HOA loop
First couple of times I put them, back scratching my head

In a 16 bit control word Hand-Off-Auto, Test, Calling, Map, Output etc.
any of the bits would flip randomly

Then when it happened again I added automatic reset to default automatic mode, an event counter & an alarm with a time stamp to log it & so production would not be affected or be mad again that I do not sleep with my phone

Also added a huge red flashing notice on every SCADA screen in the plant that cleared in 10 seconds "Paste Mixing Error"

Debug counter & alarm kept going up every couple of weeks or so when I was not there

One day it finally went off when I was in my office, I walked out to investigate in a plant as big as s football field, 2 stories

On my way up the stairs I see a welder with his ground cable clipped
to the 6" communication gutter 15 feet up in the air, watched & saw that
as he welded the bits flipped & the alarm incremented counts

Investigated further & found that the only valves & pumps affected were
isolated to one communication message between the master PLC & one of the remotes & the cable was in that gutter

Somehow those bits in the Cat 5 cable would get scrambled but
the communication packet did not error & still pass them through

The alarm code & counters are still there &
have not incremented since construction in that area was completed
over a decade ago

Had I not been lucky enough to catch it
It would have magically fixed itself
& I hate when that happens
  Reply With Quote
Old April 4th, 2022, 06:48 PM   #7
Dayvieboy
Lifetime Supporting Member
United States

Dayvieboy is offline
 
Dayvieboy's Avatar
 
Join Date: Jun 2013
Location: San Francisco Bay Area
Posts: 108
What Subnet do you put your PLC's on??

Does it ever make sense to use 192.168.xxx.xxx to put equipment on?

Using the most popular private subnet in the world
for machines seems to invite many problems and make troubleshooting harder

I was just on project having issues going online with a PLC.
Turns out they were thinking the PLC was alive but were actually pinging something that is not even a PLC a 1/4 mile & 2 buildings away.

(IT is dealing with this as private networks should never be seen public)

I have had problems several times over the years when people
plug things into an incorrect network, do not use a VLAN/NAT
or put the PLC on a the public network
& several other reasons.

192.168.1.xxx is the Rockwell option for Ethernet adaptors
when using the hardware thumbwheel.

Curious what other subnets people use for their machine networks.




Some info I found on private networks:

https://sourcedaddy.com/networking/private-subnets.html

https://www.computerweekly.com/news/...e-IP-explained

https://en.wikipedia.org/wiki/Private_network
  Reply With Quote
Old April 5th, 2022, 07:14 AM   #8
ArcReactorKC
Supporting Member
United States

ArcReactorKC is offline
 
Join Date: Jun 2021
Location: Midwest
Posts: 69
Quote:
Originally Posted by Dayvieboy View Post
Does it ever make sense to use 192.168.xxx.xxx to put equipment on?

Using the most popular private subnet in the world
for machines seems to invite many problems and make troubleshooting harder

I was just on project having issues going online with a PLC.
Turns out they were thinking the PLC was alive but were actually pinging something that is not even a PLC a 1/4 mile & 2 buildings away.

(IT is dealing with this as private networks should never be seen public)

I have had problems several times over the years when people
plug things into an incorrect network, do not use a VLAN/NAT
or put the PLC on a the public network
& several other reasons.

192.168.1.xxx is the Rockwell option for Ethernet adaptors
when using the hardware thumbwheel.

Curious what other subnets people use for their machine networks.




Some info I found on private networks:

https://sourcedaddy.com/networking/private-subnets.html

https://www.computerweekly.com/news/...e-IP-explained

https://en.wikipedia.org/wiki/Private_network
Depending on the network and customer I may still use 192.168.xxx.xxx

However most often I use the tried and true 123.123.xxx.xxx very very rarely do I have an incident where IT or some other entity is using the 123 subnet. Originally I used 10.10.xxx.xxx but had more than one collision with IT so I switched.
  Reply With Quote
Old April 5th, 2022, 08:04 AM   #9
mad4x4
Member
United Kingdom

mad4x4 is offline
 
mad4x4's Avatar
 
Join Date: Mar 2009
Location: ST CYrus
Posts: 265
Interesting little Vid. We had similar a few years ago and it was described as a "Stuck Bit" we could never get a straight answer from Stratus on what a "Stuck bit " was.
  Reply With Quote
Old April 5th, 2022, 09:28 AM   #10
Nova5
Member
United States

Nova5 is offline
 
Join Date: May 2013
Location: Northern Snowball
Posts: 563
T-Shirt

Quote:
Originally Posted by ArcReactorKC View Post
Depending on the network and customer I may still use 192.168.xxx.xxx

However most often I use the tried and true 123.123.xxx.xxx very very rarely do I have an incident where IT or some other entity is using the 123 subnet. Originally I used 10.10.xxx.xxx but had more than one collision with IT so I switched.
why are you using a publicly routable 123 address?
__________________
Problems that go away on their own mystically are not solved. They will return.
  Reply With Quote
Old April 5th, 2022, 09:56 AM   #11
I_Automation
Member
United States

I_Automation is offline
 
I_Automation's Avatar
 
Join Date: Jun 2020
Location: Detroit, Michigan USA
Posts: 1,135
Quote:
Originally Posted by ArcReactorKC View Post
Depending on the network and customer I may still use 192.168.xxx.xxx.

When I first set up my home network I, not being normal, didn't want to use the "normal" 192.168. so I picked a unique number all my own.


Then I added something (a camera I think) that could ONLY be set to 192.168.127.(2-255) and I had to reset and reprogram my router and every other device I had set an IP address on. That included other cameras, a SLC5/05 and 3 NET-ENI's for SLC5/03's. Also RSLinx ETH drivers on multiple computers and every page of 2 AdvancedHMI setups for the home system.


That was a pain (resetting an installed camera means pulling it down, reprogramming and reinstalling it) so 192.168.127.xxx it is. Also with my IP DVR for the camera's it has to be set to a specific IP for each camera, if they were not permanent IP's it would lose the camera every time it reconnected.
__________________
ivanovaautomation.com


Some Jobs Are Best Started With A Sawzall And A Dumpster On Castors
  Reply With Quote
Old April 6th, 2022, 09:34 AM   #12
Nova5
Member
United States

Nova5 is offline
 
Join Date: May 2013
Location: Northern Snowball
Posts: 563
Well... fixed to 192.168.127.x... you were kinda screwed anyway when you left the entire 192.169 range. Otherwise just open the subnet mask a bit to encompass the 127 and it coulda worked with less trouble. But I'd wager that camera got on the Never Buy Again list.
__________________
Problems that go away on their own mystically are not solved. They will return.
  Reply With Quote
Old June 17th, 2022, 07:33 PM   #13
I_Automation
Member
United States

I_Automation is offline
 
I_Automation's Avatar
 
Join Date: Jun 2020
Location: Detroit, Michigan USA
Posts: 1,135
RSLinx (not $$$ Gateway edition) can be setup to use a TeamViewer VPN connection.


Others here have done it and I checked it out, but have not done it myself yet.



This thread is one.
__________________
ivanovaautomation.com


Some Jobs Are Best Started With A Sawzall And A Dumpster On Castors
  Reply With Quote
Reply
Jump to Live PLC Question and Answer Forum

Bookmarks


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Topics
Thread Thread Starter Forum Replies Last Post
signed and unsigned guest LIVE PLC Questions And Answers 13 September 12th, 2017 01:45 AM
S7 Indirect bit addressing curlyandshemp LIVE PLC Questions And Answers 1 October 4th, 2006 02:40 AM
Tidbits, finding the bit number or index Peter Nachtwey LIVE PLC Questions And Answers 10 February 9th, 2006 04:17 AM
S7 - using Arrays with Multiplex Addressing in ProTool RMA LIVE PLC Questions And Answers 4 August 12th, 2005 08:39 AM
10 Bit Encoder Decoder/ GE90-30 Control Freak LIVE PLC Questions And Answers 6 July 12th, 2005 12:28 PM


All times are GMT -4. The time now is 07:21 AM.


.