Ping Logger

Is there such a device or piece of software that I could set up to ping the internet at fixed intervals (say, once a minute) and record the date, time, and ping duration in CSV format?

I would like to print something I could take to my ISP to suggest that I didn't want to pay for internet connectivity that I wasn't receiving. This past week or 10 days has been horrific, probably less than half of the time have I actually had a connection.

I might like to run this on SWMBO's Macintosh, which uses only a fraction of the electricity that my power-hog Windows machine consumes, since the computer would have to remain on 24/7, but I am not at all comfortable trying to do anything remotely resembling system level on that evil machine.

Suggestions?

tanstaafl.

Oh, I guess I might mention that you can enter that command into a Terminal window (found under Utilities).

The above one-liner works for me on linux, but there may be some differences in the ping output on a Mac. I don't have a mac handy, for testing.

You can obviously replace the google.com, with a URL of your own choosing (perhaps to the ISP, itself). The unit for sleep is seconds, so it'll ping google once every minute.

My output (using a shorter sleep), looks like this:

If it's worth a buck a month ($USD) to you, and you want something plug-and-play, DSLreports.com can do it for you.

http://www.dslreports.com/schedule

Alternatively, if you want to avoid perl:

Using the initial perl method (removing the loop), I've wrapped this up as an Applescript application attached to this post. When you run it, it will ask where to save the log, then run the ping every 60 seconds.

If you want to hardcode a file to write to, let me know and I'll walk you through modifying it to do so. This way this is a normal Mac app that can easily be added to startup. (Right click on the icon of it on the dock, go to options, open at login.

... uhhh, yeah.

I must confess to being a bit confused here. When I unzip the file (on my Windows machine where I know [a little bit] what I'm doing, I get a directory named _MACOSX, and another directory named Ping Test.app.

The _MACOSX directory contains a directory named Ping Test.app, and a file named ._Ping Test.app. That Ping Test.app directory contains subdirectories...aaarrggh. See the attached screen shot! (Right click --> "View Image" to make it readable)

Anyway, just what am I supposed to put on the Evil Macintosh (tm)? My plan would be to put it on a flash drive, then copy it to the Mac's desktop and run it from there. Then, after a week or so stop the app, copy the output file (via flash drive) back to my computer where I can tweak it with Excel, where I very much DO know what I'm doing.

Keep in mind that I have only laid hands on a Macintosh thee or four times in my life, and then only under duress, so explanation will have to be basic and very simple. Don't assume I know ANYTHING about those evil machines. Remember my motto: If it's different from what I'm used to, it can't be any good.

tanstaafl.

Mac applications are actually folders (as you have discovered by unzipping it on Windows). And the __MACOSX folder contains filesystem metadata for the files in the zip.

To move it to the Mac, don't unzip it on your windows machine. Just copy the ZIP to a flash drive, then unzip it on the Mac. Unzipping is built into a Mac, just double click it. Out should pop the Ping Test application, as a single file.

To "properly" install it, drag the app to the Applications folder on the Mac.

Smokeping

It's probably more effort than you want to put into it, though.

After a mis-step or two (I told you I didn't get along well with Macs) it is up and running.

When I plugged the flash drive into the iMac, I clicked on the zip file while it was on the flash drive instead of copying the zip file to the desktop. Then I ran the app from the flash drive. Not the right way to do it. I copied the app to the "Applications" directory, but could not figure out how to make the app running from the flash drive stop running so I could re-start it from the copy in the Applications directory. Finally out of frustration I just pulled the flash drive out of the USB port, amidst dire warnings about damage to file structure, eject it first next time, iMac will try and repair damage, and for all I know global warming and world-wide economic collapse, and restarted the app from the Applications Directory instead of the flash drive, and was pleased to find that it did not overwrite the output file but continued on, appending to it. Nice touch, that.

Thank you for the help.

tanstaafl.

For future reference you could have quit it a couple of ways:

- make PingTest the active app (either by click on it in the Dock or by hitting cmd-tab to cycle though the open apps)
- use the PingTest menu to select "Quit Ping Test" (or just hit cmd-Q)

(if the Dock was set to auto hide, you'd need to first move the mouse to the bottom edge, or left/right if the dock has been moved there, to make it pop up)

or

- right click on Ping Test in the Dock and select "Quit" (if you are using an Apple track pad, clicking with two fingers gives you a right click as does holding down ctrl and clicking)

In summary, very little different to how you would/could do it in Windows.

If you are willing to watch a five minute video (yes, with Quicktime, would play fine on the Mac if you don't have Quicktime playback on the PC), it may help with some of the PC to Mac differences.

http://www.apple.com/findouthow/mac/#switcher (This is the PC to Mac: The Basics one)

Running the application from anywhere is fine, the reason I said proper way is that most people put their applications into "Applications", much like most Windows users put their programs into "Program Files". It's more of an organizational thing, and less an exact operational procedure.

That's what I would have thought. But... (no doubt I'm doing this all wrong):

-->Finder-->Applications-->PingTest.app
Right-click and get the following options:
Open
Show package contents
Move to trash
Get info
Compress "PingTest.app
Burn "PingTest.app" to disc
Duplicate
Make alias
Quick Look at "PingTest.app"
Copy "PingTest.app"
Show view options
Label

none of which proved very helpful to me.

I didn't realize that there was a PingTest icon in the "dock" (is that what you call that thing taking up all the real estate at the bottom of the screen?) Yes, right-clicking on that gives me useful options.

Thank you.

tanstaafl.

Yes. It behaves fairly similarly to the Windows Taskbar. It shows running apps.

It is different to earlier versions of the Windows Taskbar however in that it can also show apps that aren't running, meaning you can use it as a launcher. Of course you can now do that on Windows as well, with the "pin" option (can't remember whether this was introduced in Vista or Win7).

I can't say I'm a great fan of the OSX Dock. I tend to launch apps using SearchLight*, by hitting cmd-space, typing the first few letters of the name of the app and hitting return.


* ok, so I actually use QuickSilver now instead of SearchLight, but it works about the same

And Doug, if you're looking for a Windows application, the only one I've been able to find is called Pinglog. Sadly, it doesn't do everything you want, and I'm never really sure if I trust Softpedia, but it's something.

I needed such an application in order to troubleshoot connectivity issues in a building-wide network. I left a laptop running that app, and logged into it remotely every once and a while to check in and see how it was doing.

Actually, I had two instances running. One was pinging the router, and the other was pinging anything on the outside, like Google's DNS server (8.8.8.8). That way I could tell if any outage was the result of the internet going down or the network. It worked pretty darn well, though I wish it gave more info than it does.

Got it covered.

With a little help from Paul Grzelak (well, actually quite a lot of help) we came up with this batch file:

:pinger
ping -n 1 google.com > tmp0.txt
echo %date:~0,5%, %time:~0,5%, > tmp1.txt
type tmp0.txt | find "loss" | cut -f 2 -d "(" | cut -f 1 -d ")" | cut -f 1,2 -d " " > tmp2.txt
echo , > tmp3.txt
type tmp0.txt | find "Average" | cut -f 4 -d "=" | cut -f 2 -d " " | cut -f 1 -d " " > tmp4.txt
paste -d "" tmp1.txt tmp2.txt tmp3.txt tmp4.txt >>pings.csv
sleep %1
goto pinger

that gives exactly the output that I want. I've attached a CSV file of the first 12 hours, and it is pretty damning for my ISP. I do NOT think it was a coincidence that the two-hour outage last night ended at exactly midnight with ping times suddenly about 25% of what they were before the outage started.

Here's what the output looks like if you don't want to load the CSV file:

9/21 21:33 0% loss 326ms
9/21 21:34 0% loss 336ms
9/21 21:35 0% loss 217ms
9/21 21:36 0% loss 242ms
9/21 21:37 0% loss 318ms
9/21 21:38 100% loss
9/21 21:39 0% loss 332ms
9/21 21:40 0% loss 208ms
9/21 21:41 100% loss
9/21 21:42 0% loss
9/21 21:43 100% loss
9/21 21:44 100% loss
9/21 21:45
9/21 21:46
9/21 21:47
9/21 21:48
9/21 21:49
9/21 21:50
.
.
.
9/21 23:57
9/21 23:58
9/21 23:59
9/22 0:00 0% loss 84ms
9/22 0:01 0% loss 74ms
9/22 0:02 0% loss 60ms

Well, sort of what it looks like. The bbs munches up the formatting a bit. Note the long ping times a few minutes before I lost connectivity. Paul's help was to give me the "cut" and the "paste" commands which are not part of Windows, and he parameterized them for me. The rest is mostly my work.

As soon as I can figure out how to do it, I will change the ping destination to be my ISP instead of google.com. I just chose google.com because I knew how to do it and they are a reliable ping object.

Anyway, I think a week or so of collecting this data should give me some ammunition to use when I visit my ISP.

tanstaafl.

This is pretty neat!
I want to run it on my work XP machine because our internet connection is horrible in the afternoons. Everyone complains but nobody does anything about it.

So, I copied this text over into a text file, renamed it .bat, and then clicked on it (it was on my desktop).
I was sort of expecting a 'pings.csv' file to appear somewhere, but a search does not bring it up.
Would you help me understand a bit more about how to activate this batch file? My comprehension is incomplete.
tmp0.txt and tmp1.txt have appeared on the desktop, though, so that's something...

You will need paste.exe and cut.exe, which Paul sent to Doug. Windows does not come with those.

Also, I suggest you rename .cmd and not .bat, as you don't want this script to run into a 16bit environment, but on the "native" XP 32bit one.

Ah. Ok.
So, something like this would be helpful?
http://unxutils.sourceforge.net/

I'll give it a go in a little while. Thanks for the tip!

edit:
sleep was also missing.
found one here-
http://www.computerhope.com/dutil.htm
I've got it running every 30 seconds.

I have now accumulated four and a half days of data, and my internet connection has been either unusable (so slow that web pages time out while loading) or completely non-existant (ping refused because of no connection) for 41.34% of the time. Even in Mexico I suspect that this is non-optimal performance.

I got lucky yesterday. I consolidated the most recent 40 hours of data onto four pages, multi-columnized with six hundred ping records per page, all conditionally formatted to color-code the four different statuses of connectivity: "Normal", "Slow", "Non-Functional", and "No Connection". I took this to my ISP which is normally a useless operation because it is staffed only by two women whose combined English vocabulary is about 10 words (note: that is MY problem, not theirs. I live in THEIR country, and I damn well better learn THEIR language if I want to communicate!) and whose total technical expertise is to tell a client if they owe money and if so, how much. But as I said, I got lucky. Their one and only English-speaking technician happened to be in the office between service calls, and I was able to speak with him. He was very impressed with my colorful printout, and 90 minutes later he was ringing my doorbell. He thought that he and his partner had found a damaged cable on the utility pole by my house and fixed the problem, but I was skeptical. As it turned out, justifiably so. So they came inside, and they hooked up some kind of signal meter to the raw cable, checked all my connections (splitter/TV/computer/modem/router), and for no really good reason the internet came back. Hmmmm... today they are going to bring me a replacement cable modem, but I think they are just taking random shots at fixing the problem. The big news, however, is that the English-speaking tech gave me his personal cell phone number and told me to call him any time, night or day, if I were having internet problems. Now that is what I call service.

We'll just have to wait and see how it goes...

Edit: I forgot to mention that after the techs left yesterday afternoon, I had internet for about five hours, then it quit again for four hours, returning on-line at six minutes past midnight.

tanstaafl.

Hmm... I'm getting carriage returns between the temporary file insertions of the pings.csv file instead of getting each series on one line.

googling for some batch file how-to or help isn't quite doing the trick for me just now. I'm weak.

Easy enough to fix after the fact. Just open the CSV file in Excel and sort by date, time. The blank lines will sort to the bottom of the file and subsequent writes to the CSV will start at the end of the data.

tanstaafl.

But you see, when I open the csv in excel, I get date and time in two columns of one line and three more lines in the first column:
date|time
loss
,
ms

If each compilation into the csv wound up on one line, I would have no issue with empty rows, but it does not.
It seems like each temp file contains carriage returns after the important stuff and those returns are being moved into the csv. I can't seem to get rid of them via tricks with cut and paste.
Appending each result directly into the csv (so that only tmp0 and csv are created/used) yielded the same thing.

I'm not sure what to say, except that it works for me. I'm running Vista, I think you said you were using XP, but that shouldn't matter down at DOS level.

I wonder if the "cut" and "paste" programs you downloaded are different from the ones that Paul gave me? If so, perhaps his parameters aren't appropriate.

If you'd like, I can send you the ones I got from Paul, so you can see if that makes the batch file work properly.

tanstaafl.

edit: Another possibility is that your ping command and my ping command produce differently formatted output. Here is what my ping command produces after ping -n 1 google.com


Pinging google.com [74.125.73.99] with 32 bytes of data:
Reply from 74.125.73.99: bytes=32 time=55ms TTL=53

Ping statistics for 74.125.73.99:
Packets: Sent = 1, Received = 1, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 55ms, Maximum = 55ms, Average = 55ms


Does that match up with yours?

db

Hi Doug-

I'm stumped, too.

The output to tmp0.txt is the same as yours.


Hmm... I copy/pasted that directly from Notepad into empty code brackets, but there are twice as many carriage returns shown in what I've pasted.
The text in Notepad looks like this-

(empty line)
Pinging google.com [74.125.224.112] with 32 bytes of data:
(empty line)
Reply from 74.125.224.112: bytes=32 time=19ms TTL=56
(empty line)
Ping statistics for 74.125.224.112:
(tab) Packets: Sent = 1, Received = 1, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
(tab) Minimum = 19ms, Maximum = 19ms, Average = 19ms
(empty line)

It seems to me that the type and echo commands are putting in too many 'new lines' than just what the data I want requires. Then the paste command just takes everything in each file (including the too-many lines) and concatenates.

If you have a handy excel trick for moving every x item from one column into another column, I could just re-organize the raw data in excel instead of worrying about the proper data accumulation technique.

~sigh~ Thanks for thinking with me.

additional:
After playing around with some of the parameters for cut and paste, I still had no different result to show for it.
The original batch file has morphed into this structure (my last attempt)
:pinger
ping -n 1 google.com > tmp0.txt
echo %date%, %time:~0,5%, >>pings.csv
type tmp0.txt | find "loss" | cut -f 2 -d "(" | cut -f 1 -d ")" | cut -f 1,2 -d " " >>pings.csv
echo , >>pings.csv
type tmp0.txt | find "Average" | cut -f 4 -d "=" | cut -f 2 -d " " | cut -f 1 -d " " >>pings.csv
echo , >>pings.csv
sleep 3
goto pinger

This structure removed the paste command entirely, relying on addenda to the .csv file each go-round.

That may be something that Notepad does.

Here are three screen shots. The first is taken at DOS after the ping -n 1 command, and then displaying the output file.

The second is what the output file looks like opened up in Notepad.

The third is what the output file looks like after copying in Notepad and then pasting into MS Word. This third file is also what it looks like when pasting from Notepad into the empeg bbs. (I deleted the superfluous line feeds from my previous post.

In any case, I don't think the extra line feeds in the display of the output would matter, because the way the batch script works is that it "finds" a line with a key word, then acts on it.

I've asked Paul if he has any ideas...

tanstaafl.

It's not DOS.

Yes, I know. I'll repeat the mantra: "There is no DOS in VISTA". But whatever it is is a good enough imitation to keep me happy.

tanstaafl.

Yes, that was my thinking, too.

I was half expecting Excel to do a better job importing the data- especially when I specifically tell it to only mind the commas, but it's not been a successful route.

The good news is that on the grand scale of things, this issue is just peanuts. Kinda fun to dink around with, but nothing to really worry about.

Well, it was entertaining to play around with this stuff for a while.
Unfortunately I never tweaked it well enough to have any meaningful data that would help with the network issue at work.

Moot point now- they laid me off today. So I don't care much now if they ever sort it out.

Many thanks to Doug for sharing his trials and tribulations from the networked netherlands.

Onwards! Into the unknown!
I wonder what happens next?

Ouch -- very sorry to hear that. Was it something you knew was coming eventually, or was it a surprise? Not that there's any good way to get let go, of course.

Good luck with your next move.

It took some patience and perseverance, but I did get some results.

My English-speaking tech was kind of a dead end. His job, and his entire focus, is working with cables, connections, modems, and routers. "If your only tool is a hammer, everything looks like a nail." His focus was on what he was used to fixing (noise in the line, bad connection on the pole outside the house, reboot the modem and router, etc.) and I could not get through to him that the problem was not local.

Why did I think this? Because even when pinging Google showed absolutely no connectivity, I could always ping the gateway IP address and get practically instantaneous (<1ms) returns, and the cable TV never faltered.

After presenting him with a several printouts demonstrating this, he turned me over to the engineer at the office. He had about the same amount of English as I had Spanish, so we were able to more or less communicate, and he found the printouts informative and intriguing, although for the first two weeks he was stuck in the same rut as the installer, looking for local fixes. He, too, gave me his cell phone number!

I don't know if it was coincidence, or if the engineer finally put pressure on upstream (where I believe the problem originated) but at 3:30 (more or less) on October 22, following a nine and a half hour outage, the internet came back on, but with a difference. The ping times which had been running (on the occasions when there were any pings at all!) an average of 50-60 ms, were now running 70-90 ms, about a 50% increase, but still perfectly acceptable. The big difference, however, was an increase in reliability, from about 50% when I started logging to more than 97% since October 22. It would be more than 99%, except there was a [probably] planned outage of 2.5 hours on October 25. Other than that outage, all of my glitches have been single, occasionally two in a row, lost packets scattered randomly maybe 10 times per day. I'm told this is normal.

My thinking is that there was a problem with some piece of equipment upstream from the ISP, evidenced by gradually increasing ping times getting up into the 300-400 range before total loss of connectivity; then sometime later, frequently at the top of the hour, the service would return rock solid (for a while) with pings in the 50's, then after several hours (sometimes just a couple, sometimes 15 or 20) the cycle would repeat.

I think someone, somewhere upstream, was being reminded by their watch beeping on the hour, and going to "check the line" and resetting something. Finally they either replaced a failing piece of equipment, or possibly changed the routing so the ping to Google.com takes some extra hops, which accounts for the 50% increase in ping times, and the huge increase in stability. Of course, this is just surmise on my part, based on invincible ignorance of how the internet actually works, but it seems plausible to me.

I am now pressing the ISP to give me six months of free internet service as compensation for several months lack of service (the problem was there for at least one month before I began logging, possibly more) and for the thorough documentation I provided. The engineer has agreed that this is reasonable, and said he would work on it. I'm not holding my breath.

Since I doubt that anyone here is interested enough in my tribulations to want to look through 51,618 data points (that's what I've collected so far) I am attaching a screen shot, showing just the statistics for the first week and the last week (more or less) of the data collection. And if there are any Excel freaks among you, I am attaching the Excel template into which I paste the CSV data from the PingLogger batch file. There are a couple of neat things in it, particularly cells D2 and F2, and the conditional formatting in column C. [Owww, my arm hurts from patting myself so much on the back ]

tanstaafl.

I was thinking it would happen in spring, after a big project we are (I was) working on.
Thanks for the luck!

Here's another screen shot (not that anybody is likely to care) showing what my internet service was like, day by day, in September, October, and November.

The improvement in connectivity has been dramatic, the off-line periods have decreased by a factor of 35 in November as compared to September.

tanstaafl.

Looking good. Somewhere archived, I have a web page I hacked together to do a very similar thing for my ISP in 1999. The support staff were so amazed that a customer was monitoring it so closely, they used my site to measure the outcome of some tweaks they tried. Being that it was a microwave based wireless solution, sometimes those tweaks involved driving up the mountain and reaiming antennas and quadrant boundaries.

Sadly it all went downhill when Sprint bought the local company out, but it still came in handy for me to use to decide when I was going to switch away. Sometimes companies out there do appreciate having a knowledgable customer interested in working with them to fix a problem.

I can beat that

The purple is downtime.



Fortunately, a new router appears to have fixed it, for now.

So exactly wtf happened there? New router at your end or their end?

Power notch appears to have killed my old router (but leaving it with enough magic smoke to appear to be working); I've borrowed a replacement from one of the guys at work until I get my FTTC upgrade installed (which requires a VDSL modem, rather than an ADSL2+ modem).

Impressive!

A&A have provided the best ISP user experience I've ever had.

Agreed. They've been nothing but excellent while diagnosing this fault.