Interesting power glitches / Controller off line

Hey All so this is interesting.
I noticed while doing some repetitive small jobs over the last couple weeks that about 4 times in 70 the cutter went off line with a cutter not responding message box pops up. Pressing the connect will not work. So after trying all kinds of things to reconnect, the only thing that would allow it to reconnect was to reboot the cutter.

In all 4 cases it was asking for a manual router speed change to zero in prep for a bit change. and in all 4 cases I flipper off the Router and the Vacuum switch at the same time. And it happened with a very small shop vac and a larger one. The Vacuum is on one circuit 15 amp 14 awg The Cutter, Router, Computer are all on another Circuit 15amp 12 awg. Vacuum hoses are grounded.
It has not done it yet when I turned one off at a time, (so far), but that could just be a coincidence so I am not yet convinced the vacuum load has any thing to do with it and besides, it is on a separate circuit.
I have an out of the box setup 4 XXL PRO. Carbide Router and BitSetter. No BitZero, or BitRunner
The computer does not need to be reset. nor the com line or any thing else, it came down to having to reboot the Controller.

Yes I put a support ticket in and we are going back and forth. :slight_smile:

Today I had a bit of time at the end of the day so I would try something.
I went in to jog, I put the router in the center, put the speed to low and hit the Y jog button and held it down. The router is slowly moving and at the same time I start flipping the router and vacuum on and off together .
Wow it messed up several times. always on the off cycle, I am guessing at 1 in 20 ish times. I really did have that much time
Some times it just stopped jogging and I had to release and press the button again
Other times the cutter disconnected and needed to be reset,
A couple of time I could try to reconnect but it was not pretty, . all kinds of weird things happened. IE it would be lost, so you init it and it would go to home but after that it should come to the front, it did not, I tried a rapid move and all of them returned it to home. so to me it kinda recovered but the memory was messed up. it was responding kinda but certainly not useable.
Once it was messed up nothing would fix it but to reset the Cutter.
It might also be when on the AC cycle it happens so it just looks really random

All of this IMO points to power spikes getting through to the the controller. It is the weird things happening that really convinced me that it a controller issue. Going off line could be anything I guess but doing strange things seems to be messed up the controllers memory.

I see other people have mentioned having some off line issue in the middle of a jobs, did this happen when the Router was turned off or another machine in the shop ???
So I thought I would share my particular findings maybe it might help
I see people trying possible work arounds and grounding but the Cutter or Cutter P/S IMHO should reject any power spikes.

Some excellent detective work there. If I may I will add a little to your findings, and this is perhaps generic and not specifically targeted to the CB3D controller…

CPU brownouts are just as bad and these are the events that really cause a processor to really do strange things. When a device that has a big motor turn on for a split second the windings of the motor are a direct short until the magnetic field builds up. You see this when lights dim a little as something starts up (AC unit, big dust vac and so on). These are “inductive loads” as opposed to “resistive loads”.

Most power supplies have enough capacitance to survive these events and indeed the CB3D controller has some big caps to also help with this. But once in a while a certain string of events will create the prefect condition for this to occur. Generally the only way out for the processor is a complete reset to a known condition.

You need an oscilloscope to see these events as they occur quickly.

Just some more food for thought . . .

2 Likes

I have my unit plugged into a UPS. No issues since then.

2 Likes

Interesting, Thanks. I assume you had problems before ?

So I put in a bunch of work today and I believe I found the problem and want to share a couple of things
In my system I have a torsion box table and all my wires are in the box cavities along with the Pi P/S and the Cutter P/S. I have a custom switch panel to turn 120vac off and on.

I put 10 ish ferrite cores all over and I could still recreate the EMI / RF issue.

After tons of testing as it is very random. well it appears random, I found out that when I pulled the cutter P/S out, I could not reproduce the failure. So more work and here is what I found. All the AC wiring are separated from the low voltage, however the Cutter P/S is in the 120 VAC cavity and then the DC output takes a 90 and heads to the low voltage pathway. the Vacuum 120 VAC was right up against the cutter’s P/S. OK so when I moved it into the next cavity, I could not reproduce the error. Remember it is random ish. I put it back and there were the errors again, I tried several combinations but this was the best one I could reproduce.

To Summarize 10 ish Ferrite cores and I moved the Cutter P/S from being in parallel with with the vacuum 120 VAC line

So that great but here is my issue. the rules are that equipment is not allowed to generate harmful noise and that is measured at, lets just call it X to keep things simple as I don’t want to get into engineering here. So equipment is not allowed to create more then X amount of noise but it very much can create noise, The flip side is Equipment must also be able to reject X amount of noise.
So as much as we are trying to eliminate the source of the noise here, IMHO The cutters controller and P/S seem to have an issue with rejecting the allowed amount of noise. Plus there is an environmental twist here to as we are not operating these on our kitchen tables so the amount of noise equipment must be designed for is not retails noise, it must be commercial noise.

Issue two, I want to be as positive as possible with C3D as I love the product and the people, however I contacted support which was frustrating me as I was willing to help research this EMI issue. Lot of us have experienced it, so I was willing to put my time in to trying to help narrow it down. I have a background in Industrial electronics and Computes. What I got was he said I have an environmental issue and that is was my issue to keep trying things until I found my problem, good luck. :frowning: Well I guess everyone has a bad day :slight_smile: I thought seeing as I could repeat it on a regular bases I could really help narrow it down. Oh Well, And Please I don’t want to say the tech support was bad. as there are lots of good people helping.

I love the car analogy. so this is like me saying my car dies when I go over a bump on the road, And them saying well it works on our test track. No you cant slam you car into a large pot hole, but it should handle normal bumps in the road. And the answer cant be well it is the roads fault and find a better road or route to take :slight_smile:

In My opinion the Engineers need to look at rejecting commercial environmental noise to the proper standard. It seems to get into the 5 volt line of the processor, For me it does not seem to get in there via the data line although I have seen recommendations to use the original USB cable with the ferrite core on it, I have seen Router brush issue, ETC… So it appears the noise can come from different sources. in my case it was a AC Line in parallel with the P/S. Although the P/S should reject this noise. FYI the Pi power Supply is in the same spot and it rejects the noise fine.

I also find it interesting that for software issues C3D is awesome and works with us back and forth to narrow down the issues, I can only assume the hardware design is outsourced.

I hope this makes it way to C3D to help narrow down the EMI / RF sensitivity issues. Prove that the noise is of an acceptable level and should not make it through to the processor. And
to help others as this IMHO is all about shielding the sensitive controller for normal noise. So you 9have to look at it the other way around.
I still have an open ticket in with C3D and I have still offered my help and notes.

Hope this helps

@rbreininger, So interesting on adding the UPS, Although it has battery backup it also has a whack load of EMI / RF filtering in it.

In my case above it was coming in via the P/S so it would have been introduced after the AC line filters. I have seen all kinds of fixes and different ways the EMI / RF is getting in, and we seem to be going a bit in circles looking for an answer, That is why I wrote the above as IMHO you have to look at it from the other side. the equipment is over sensitive to EMI /RF so it could be all kinds a different ways but what is common is it gets into the processor Vcc+ line and spikes the processor. This IMHO is proven as it doesn’t always just cause the cutter to stop, it also causes memory glitches, the cutter is still running but just gets messed up a bit. Losses position etc …

Again I hope that this helps and we can all share to help narrow this down to a great solution or root cause

A couple of things that bugged me:

When you wrote “separate circuit”, do you mean each of those “separate” circuits have their own circuit breaker or that they are only plugged into different plugs?

When one speaks of “EMI/RF”, one is usually not speaking of low frequencies like power line frequencies. A UPS will transfer to battery (with a click of an internal relay) when there are power line variations beyond its set limits. Here’s some additional information.

Agree there’s a lot of work to be done. Far too much blame is placed on static electricity. Static electricity doesn’t go anywhere, otherwise it wouldn’t build up in the first place.

I experienced disconnects with heavy loads on my Makita. Bigger load, bigger EMI. Chokes helped a lot, but I would still get occasional issues.

Since switch to the spindle, things have been much better. I think proper shielding would go a long way for whatever the next product is.

My UPS has saved me many times.

The CNC is on the same circuit with my miter saw which occasionally will trip the breaker. The saw goes down along with the dust vacuum. The CNC keeps running with absolutely NO issues. Reset the breaker, turn on the dust and go merrily on my way including using the miter saw.

@CrookedWoodTex, Hi Yes two separate Circuit Breakers

As for the UPs, it was mentioned that that it helped in one case, I was pointing out that UPSs helps with power lose or brown outs however they have a fair bit of EMI / RF filtering, so adding one may help however maybe it was just the filters in the UPs that helped.
A EMI filter or Ferrite cores may work and is a lot less expensive.

I see there has been lots of talk about losing communications as that is the error we get on the screen however in my troubleshooting it has always been the controller that gets messed up. Non responsive, or the memory gets glitch and it starts doing strange stuff. Like losing its position, goes off in some random direction … To me the processor got wacked.

I see the controller board has some large caps, I assume these are filter caps for low frequencies. I have not taken the controller out to reverse engineer it :slight_smile: , Normally there should be fast acting capacitors and cokes to deal with high frequencies, I have no idea if these are there.

In my case and IMHO, If you turn equipment off on high current draw, at the top or bottom of the AC wave form, it may generate a EMI / RF spike. it appears random as it depends on where on the AC wave it accrued And again in my case, it was parallel coupled in to the controllers P/S.

The EMI can come from many places, what is consistent is it would appear the controller is quite sensitive to it, Knowing that as the possible root cause it allows us to look at things from the other side. Lots of equipment generate noise ( that is normal) and we can try to trouble shoot by looking at that to the controller or we can look from the controller out by figuring out ways to shield the controller.
Maybe by looking at it this way we can find a solution that works for all “noise sources”

Maybe looking at it this way, it is as simple as simple a a couple of wraps of the power cable around a ferrite core ( a P/S EMI Filter ) and Making sure the 5 Vcc is decoupled from the Data cable ( a common engineering oops that can exposes 5 Vcc to external noise ). So in other words, how to protect the 5 Vcc in the controller verses removing all the possible noise conditions

Again I hope this helps us get to an easy solution :slight_smile:

I’m guessing that you don’t have many local power outages, which is why a UPS should be powering your system. It will also stabilize your power for those times that you are describing. Its your choice, but when I get a quick blip in the local power — I just keep on cutting wood.

Ah you need it for other reasons :grinning: