[The Most Loathed] learns....slowly

[Endor]

Just ordered one  

 

Watched a vid on the media player, doesn't actually look that good to me and I'm pretty happy with my PS3 + Chromecast. Reckon I'll set it up as a server or maybe run an emulator on it. I'm not bad with Perl, never tried Python so that will be interesting. 

 

I don't know much about Hadoop, what are you going to use that cluster for? I assume it's something like customer analytics? I tell this story in the context of big data (and it's evil powers!) sometimes to non-IT  folk to pique their interest. 

[Rooks]

That reminds me that I need to figure out what to do with mine at home. I'm thinking I'm going to use it as WiFi extender to hopefully get better coverage in my detached garage. 

 

Either that or a portable Hack box and put that portable version of Kali Linux on there. But that would take some big commitment (to actually learn how to do some of that), so we'll see if I go that route.

[The Most Loathed]

On 8/30/2017 at 11:44 PM, Endor said:

Can you run other stuff on it as well as the media server or would that software consume it?

 

I just bought some concert tickets and by chance they came with a $30 voucher for the site I was going to buy the Pi from so I think that's sealed the deal  

Yeah. A free or virtually free Pi, definitely get it.

 

I've asked similar questions about the capacity of the Pi to do multiple things. It would vary based on the script. If it goes out once a minute and does the scraping, you will have quite a bit of capacity left. If your scraper runs constantly, with no sleep cycles in it, it could consume your processor 's entire bandwidth. I would imagine that running the scraping and a media center would occasionally cause conflicts.

[The Most Loathed]

On 8/31/2017 at 2:19 AM, Endor said:

Just ordered one  

 

Watched a vid on the media player, doesn't actually look that good to me and I'm pretty happy with my PS3 + Chromecast. Reckon I'll set it up as a server or maybe run an emulator on it. I'm not bad with Perl, never tried Python so that will be interesting. 

 

I don't know much about Hadoop, what are you going to use that cluster for? I assume it's something like customer analytics? I tell this story in the context of big data (and it's evil powers!) sometimes to non-IT  folk to pique their interest. 

 

On 8/31/2017 at 8:15 AM, Rooks said:

That reminds me that I need to figure out what to do with mine at home. I'm thinking I'm going to use it as WiFi extender to hopefully get better coverage in my detached garage. 

 

Either that or a portable Hack box and put that portable version of Kali Linux on there. But that would take some big commitment (to actually learn how to do some of that), so we'll see if I go that route.

Sorry, I just saw that these comments came before my last reply. 

Lifehacker has decent articles for all of the things you guys are talking about. I wouldn't use them as my only source but they are a good starting place:

Retro Pi

Portable Kali

 

@Endor - I'm a fan of Python but like a languages, if you're not going to use it it's probably not worth wasting any time on. I don't know the nature of your work but looking at your log, it sounds like you don't really have the time to learn it right now. It'll always be there, if you have time in the future, play with it but until then, it's just out there.

 

@Rooks - I think you were one of the fans of the Retro Pi idea. You could easily start there. 

[The Most Loathed]

What a week.

 

Monday my boss learned that I'm changing jobs. She proceeded to go into a slow panic about it. It was annoying but nothing I couldn't manage

Tuesday we noticed a big angry lump on our old dog's leg. We had never noticed anything like it before. I called the vet immediately

Wednesday I took the dog to the vet. They had to poke the lump with a needle to see what's inside. Lots and lots of blood. She has a mast cell tumor. My parents arrived from Iowa. We took them to our first St. Paul Saints game. We enjoyed it and will likely do it again sometime.

Thursday our dog was doing worse. We started trying to give her Benadryl to manage inflammation. We took my parents to the State Fair for a half day. I made fajitas for everyone that night. Between the baseball game and the fair we'd had plenty of junk food so I figured something cleaner would help. We ended the evening with the dog starting to drip blood wherever she walked. Laura and I took turns that night sleeping on the floor by the dog to try and keep her from licking the growing wound.

Today, Friday, we booked time at the vet and joined my parents for breakfast. Afterward my parents headed back home and the dog looked even worse. We called the vet and jumped in the car and rushed over there. We just got back. We left Bernie with them to try and clean up the wound and do some more assessment. It's known that these tumors can do this and it can be a hard tail spin to get them out of. I don't know her odds but it seems like we could lose her this weekend. If we don't she may have months or years to go. 

 

@Endor - I forgot to reply to a your Hadoop question.

My home Hadoop cluster is just for learning although my city publishes all of their police stop data so I'm considering seeing what I can do with it. Hadoop is especially good for huge volumes of data that don't have a ton of attributes or where you can narrow your focus to a few attributes. If I do anything cool, I'll keep you posted.

The Hadoop cluster at work is very interesting to me. I knew going in how they were using it which is that they basically dump all of their data into the server then use it to structure the data for other people to consume. Interestingly, this isn't really what Hadoop is best at. However, I've been asking myself, what is best at this? I haven't found an answer so I'm still doing homework on that. I'm kind of hoping that in the coming year I can actually do something that is worthy of Hadoop's distributed computing power.

It's interesting that you linked to that article, I know those people. I worked at Target for about 5 years and was hired shortly after that article was published. I can tell you that they didn't use Hadoop, they used a customer segmentation model that was built by a contracted firm. I'm personally friends with the guy who is working on similar for their website. In fact, we're playing board games next week. 

[dayzee84]

I hope your dog is feeling better!

[Rooks]

27 minutes ago, The Most Loathed said:

@Rooks - I think you were one of the fans of the Retro Pi idea. You could easily start there. 

 

Yeah. That idea occurred to me, but I think it would make the better half upset. She's not a huge fan of video games, and if I'm truthful, I use them as an escape. So it's better to just not have them around to tempt me.  

 

17 minutes ago, The Most Loathed said:

Thursday our dog was doing worse. We started trying to give her Benadryl to manage inflammation. We took my parents to the State Fair for a half day. I made fajitas for everyone that night. Between the baseball game and the fair we'd had plenty of junk food so I figured something cleaner would help. We ended the evening with the dog starting to drip blood wherever she walked. Laura and I took turns that night sleeping on the floor by the dog to try and keep her from licking the growing wound.

Today, Friday, we booked time at the vet and joined my parents for breakfast. Afterward my parents headed back home and the dog looked even worse. We called the vet and jumped in the car and rushed over there. We just got back. We left Bernie with them to try and clean up the wound and do some more assessment. It's known that these tumors can do this and it can be a hard tail spin to get them out of. I don't know her odds but it seems like we could lose her this weekend. If we don't she may have months or years to go. 

 

Ugh... sorry to hear, sir. That just flat out sucks. Hopefully everything turns out ok.

[The Most Loathed]

5 hours ago, dayzee84 said:

I hope your dog is feeling better!

 

4 hours ago, Rooks said:

 

Yeah. That idea occurred to me, but I think it would make the better half upset. She's not a huge fan of video games, and if I'm truthful, I use them as an escape. So it's better to just not have them around to tempt me.  

 

 

Ugh... sorry to hear, sir. That just flat out sucks. Hopefully everything turns out ok.

Thanks. We've retrieved her from the vet and she seems to be in good spirits. The vet was optimistic and none of the blood work or x-rays indicated additional tumors. She's on a pretty good dose of steroids for now. Now we play the game of which is worse, side effects or disease. Her leg is cleaned up and wrapped so hopefully the wound itself will resolve.

 

 

For your Pi, I can understand why a Retro Pi may not be the right thing. There are a million options out there but until one strikes you, it's hard to feel like it's worth the effort. I'd go to the trouble of at least loading your OS and getting a VNC set up so when you are ready you can just plug and play.

[Elastigirl]

Hope your dog feels better quickly

[Sloth the Enduring]

Poor puppers. I hope she heals soon.


Sent from my iPhone using Tapatalk

[LouCarJo]

Sorry to hear about your dog - hope she gets and feels better soon.

[Bearlee]

Sorry about the dog. Would that be Clementine? Hope things go well for all of you.

 

I love following NFers, I always learn something new. Thought "Hadoop" was a word you made up but looked it up and learned something new. So last month my favorite new word was "syzygy" (think this would score high in Scrabble) and this month it's Hadoop.

[The Most Loathed]

On 9/1/2017 at 6:27 PM, Elastigirl said:

Hope your dog feels better quickly

 

On 9/1/2017 at 7:21 PM, Sloth the Enduring said:

Poor puppers. I hope she heals soon.

 

On 9/2/2017 at 0:34 AM, LouCarJo said:

Sorry to hear about your dog - hope she gets and feels better soon.

 

20 hours ago, Bearlee said:

Sorry about the dog. Would that be Clementine? Hope things go well for all of you.

 

I love following NFers, I always learn something new. Thought "Hadoop" was a word you made up but looked it up and learned something new. So last month my favorite new word was "syzygy" (think this would score high in Scrabble) and this month it's Hadoop.

 

Thanks. Bernie is doing better but we don't know how much better. We had to take her back to the vet again yesterday because she was bleeding through the bandage that they had placed. They said things looked "good" but I mean, how good can an inflamed tumor that is still bleeding after three days look? We got her started on her steroids and antibiotics. She's still not eating but that's not that abnormal for her. She at least wants to walk again. We take her back on Wednesday to get the bandage swapped and have the situation reassessed. Maybe we're coming out of the woods now.

Clem, I think has an ear infection again but that's really not that big of a deal.

 

I've been a bad NFer of late. I haven't kept up on your threads. I know that @Elastigirl had an amazing eclipse trip, @Sloth the Enduring is going to die doing Cyclocross and that @Bearlee continues to be and awesome scout hero, and @loucarjo is trying to enjoy her holiday but finding it hard at times. I'll do better at keeping up.  I have excuses but let's leave it there.

 

No workout yesterday or today. Yesterday we were dealing with dog issues, today I think that we were just happy to not have to do anything

 

I have taken advantage of the time to do some coding. I created a Brandon Sanderson Markov Chain generator. Basically I took the first three chapters of Oathbringer, which Tor is publishing on their website and fed it into a system that then kicks out sentences that are statistically similar to Brandon's. Unfortunately I fell short at actually making it accessible via a webpage. I still could but having achieved my primary goal I'm running out of motivation to keep going.

 

I spent some money today. I bought 4 more Raspberry Pis, spacer screws, a pack of 1 foot ethernet cables, a pack of USB charging cables, and a couple USB charging hubs. The plan is to build out a little group of Pis that are physically together but also share a network connection. I'll turn two into Hadoop nodes and at least one into a Spark node and play with them as I study up for the new job.

 

Tomorrow is supposed to have a workout in it for me. We also need to go grocery shopping. More planning for the week then.

[Elastigirl]

Never heard of a Markov Chain before. Googled it, and now I have a semi understanding of it. Anyway, sounds like a fun experiment. 

[Endor]

On 02/09/2017 at 3:03 AM, The Most Loathed said:

My home Hadoop cluster is just for learning although my city publishes all of their police stop data so I'm considering seeing what I can do with it. Hadoop is especially good for huge volumes of data that don't have a ton of attributes or where you can narrow your focus to a few attributes. If I do anything cool, I'll keep you posted.

The Hadoop cluster at work is very interesting to me. I knew going in how they were using it which is that they basically dump all of their data into the server then use it to structure the data for other people to consume. Interestingly, this isn't really what Hadoop is best at. However, I've been asking myself, what is best at this? I haven't found an answer so I'm still doing homework on that. I'm kind of hoping that in the coming year I can actually do something that is worthy of Hadoop's distributed computing power.

It's interesting that you linked to that article, I know those people. I worked at Target for about 5 years and was hired shortly after that article was published. I can tell you that they didn't use Hadoop, they used a customer segmentation model that was built by a contracted firm. I'm personally friends with the guy who is working on similar for their website. In fact, we're playing board games next week. 

Ha! What a small world! Well tell him I love the story and very clever use of technology. I guess he'll have modelled up some crazy code that analyses your past moves, predicts your strategy and then crushes you at the board games!   

 

After thinking about your traffic stop data, I like the idea of analysing some public domain data to create something useful/useable. Would be a big project for me, I haven't programmed any web site stuff for about 10-15 years, I can imagine it's moved on a lot. 

 

My Pi just showed up so I have it on PiP setting itself up while I'm doing some other work. I'm way too excited.  

 

Glad to hear your dog is on the mend, fingers crossed all goes well. 

[The Most Loathed]

2 hours ago, Endor said:

Ha! What a small world! Well tell him I love the story and very clever use of technology. I guess he'll have modelled up some crazy code that analyses your past moves, predicts your strategy and then crushes you at the board games!   

The guy is super smart but no, it's not crazy modeling. He and I have the same observation, past behavior is overwhelmingly the predictor of future behavior. I did similar in healthcare, the number one predictor of whether you will have a heart attack is whether you have had one before. Similarly, the number one predictor of whether you will buy a dog food is if you have bought it before. Most recommendation engines know this and will recommend things you already bought or things sharing a large number of attributes of with things you already bought. The most complex models that are effective is the "people who bought that thing also bought this thing". Beyond that, you recommend what the company is pushing and do a little bit of customer segmentation which is just demographics grouping. Sadly, there's no real magic in it. 

When I was doing that stuff, the holy grail was "converting" a customer. How do you get a person who has never bought milk at your store to buy a gallon of milk and keep buying gallons of milk. There were hundreds of "Success" stories but never ever stood up to serious scrutiny. Even if you show a bump in sales, it's extraordinarily hard to say why and regression to the mean is a rotten bastard that will always get you. 

My personal success story in that area is a bit of a cheat. I worked in the pharmacy area. I personally coded up a big program that I ran every day. It combed through our data and found everyone who was on a chronic med who should be missing dose and told the pharmacist to call them. Other companies do the same thing but when we priced it out it was going to run $10,000,000. I did it with a few hundred hours of my own labor. No guest conversion but better sales and they were sustainable. 

 

On 9/3/2017 at 7:53 PM, Elastigirl said:

Never heard of a Markov Chain before. Googled it, and now I have a semi understanding of it. Anyway, sounds like a fun experiment. 

It was fun. It was just a toy but it was fun.

[Endor]

1 hour ago, The Most Loathed said:

past behavior is overwhelmingly the predictor of future behavior

I can agree with this, how often do people change what brand they buy of supermarket products? Not very often.

 

1 hour ago, The Most Loathed said:

My personal success story in that area is a bit of a cheat. I worked in the pharmacy area. I personally coded up a big program that I ran every day. It combed through our data and found everyone who was on a chronic med who should be missing dose and told the pharmacist to call them. Other companies do the same thing but when we priced it out it was going to run $10,000,000. I did it with a few hundred hours of my own labor. No guest conversion but better sales and they were sustainable.

I think I'd find this sort of work very rewarding, you get to see some definite results from your actions. Also in this case, you're not only pushing sales but also doing people some good helping them to remember their meds. 

[The Most Loathed]

11 hours ago, Endor said:

I think I'd find this sort of work very rewarding, you get to see some definite results from your actions. Also in this case, you're not only pushing sales but also doing people some good helping them to remember their meds. 

It was. Unfortunately it was a few months out of my life but a bright spot no less. I hope to make more of them in the future. 

[The Most Loathed]

Tuesday

Left Snatch ladder 1-> 10

Box Jump ladder 1->5, 5->1

Right Snatch ladder 1-> 10

Battle Wagon ladder 1->5, 5->1

Assault Bike 1:00

 

Battle Wagon is the 4 wheeled wagon that they have loaded up with 400lbs. It has three gears in it, I left it on low. I like the idea of it but quickly grew kind of tired of it. There isn't a strap or handle or anything so as you pull it out you take tiny, mincing steps. Rolling it back is still a good workout though.

 

Wednesday - off. We took Bernie for a follow up on her knee.We got to see it, there is still a huge tumor there, the steroids aren't making it better but theoretically will keep it from getting worse. We hoped she wouldn't need a bandage anymore but apparently the wound is still there but much smaller. So they rebandaged it and sent us home, told us to come back next week.

 

Thursday - off, unplanned. We're both dragging after broken up sleep with the dogs, Laura's having a crap week and appears to be coming down with something. We had Indian for dinner and I reinstalled the OS on my Raspberry Pi in prep for setting up a multi-node Hadoop cluster this weekend. I also played around with an online contest to use some of the Titanic's passenger data to predict who would survive out of the rest of the data. Kind of morbid but an interesting challenge. 

 

Friday - today - took other dos (Clem) to a new vet, much closer to home than Bernie's vet to have her obviously infected ears looked at. all went well, not much to add. Meds for her for a couple weeks but no big deal. 

 

This weekend:

Laura seems to be starting a cold. I felt like my throat was a little tickly during cardio this morning so maybe I am too. regardless, things will be pretty quiet. I was supposed to play Call of Cthulhu tonight but the GM had to postpone. 

Tomorrow we need to do some grocery shopping and I'm supposed to meet up with someone online for a little programming in the afternoon. 

Sunday is quiet.

 

[Endor]

Day 1 - I hate python

Day 2 - i love python

 

so roughly par for the course when I'm learning a new language!!

[The Most Loathed]

1 hour ago, Endor said:

Day 1 - I hate python

Day 2 - i love python

 

so roughly par for the course when I'm learning a new language!!

hahaha. That was a quick turn around. Tell me what you're up to and what's giving you trouble. @RisenPhoenix did an awesome job working through Automate the Boring Stuff. I need to find my copy and get back to it. In the meantime I'm working on pet projects and know someone who is working through HackerRank so I've been following along with him. 

 

The weekend was pretty quiet. We did our usual chores plus hit up our favorite apple orchard for some apples straight off the tree. It's hard to beat those. I spent a large chunk of my weekend setting up SD drives to get my RPi cluster running only to have my network cables delayed in the mail until Tuesday. So right now I have two of my cluster online and don't have hadoop actually running on them because I want to have them all running before I start that process. 

I spent some time on the phone with a pretty impressive programmer. He's on a Python forum that I participate on. He was trying to get a few people to get together and work on some open source projects. I volunteered despite being one of the poorer programmers on the forum. So he introduced me to the program and gave me a walk through of some of the needs of the project and eventually got two assigned to me. So, I also spent some time just reading the code and trying to figure out where I need to jump in. 

 

My obsession for this week is, how do I do better at documenting the work I do (see previous talk on "billable hours") but also, all the random threads that spin off that could be new ideas, things to research, questions to answer and so on. If you have a method that works, let me know. I know a couple of us have tried BuJo but as far as I know none of us have stuck with it long term.

 

This week

• Monday - short workout, just some conditioning I think. Afterwards we are taking Bernadette to get her wrap taken off her lg finally. 
• Tuesday - gym, may do squats but go a little light
• Wednesday - off. time to work on Pi cluster
• Thursday - deadlifts. I kind of wanted to go the local Python meetup but it's just not going to happen this week
• Friday - game night. We're having friends over for dinner. We saw that a local school is having a food truck rally so dinner will be food trucks then back to the apartment where we will be playing Truck Off among other things.
• Saturday - no plans this weekend but Laura will be out of town next weekend so I would imagine that there will be some extra running around to get ready for that. 
• Sunday - see Saturday

[Endor]

40 minutes ago, The Most Loathed said:

Tell me what you're up to and what's giving you trouble

I was thinking of something practical and useful for me to apply python to since that is generally more interesting and motivational for me than follow the bouncing ball courses. So I decided to rewrite my already working Perl code to scrape a website in Python. Well looky looky my perl modules are available in Python, how fucking cool and easy is that.....

 

After about 10 hours or so dicking around loading modules and wondering why the fuck the code thinks they don't exist in Python when 2 seconds ago I loaded them...oh wait.....my nice little coding software included in Raspbian is Python 3 based and all the cool shit I know and love is Python 2.x and downloading them doesn't mean dick in my nice free P3  interpreter....

 

So now I'm listening to my new Indian friend on youtube telling me the abc's of git and how I get that shit working....

 

Right now I can spawn a browser and login to a website with Python.....next challenge is identifying the form i want and clicking the right buttons so i can suck out the data i need,,,,,all piss easy in Perl but the same form name doesn't work in Python...fuck knows why at this stage!!!

 

[Endor]

54 minutes ago, The Most Loathed said:

He's on a Python forum that I participate on

Link pls? PM if you prefer. 

[Endor]

56 minutes ago, The Most Loathed said:

how do I do better at documenting the work I do

I'ma consultant so my hours are reasonably well monitored but not anally so. I have tried a few things and surprisingly to me the best method is a paper book, i just find it faster and more easily referenced to note things down and easier to see a long period of time in a quick  snapshot on paper. How granular do you need to go? We have software that will monitor to 15 min interval but generally we bill in 4 hour blocks so not had to go that far yet yet.... thankfully.

[RisenPhoenix]

1 hour ago, The Most Loathed said:

@RisenPhoenix did an awesome job working through Automate the Boring Stuff.

 

I mean, you say that, but right now I'm not sure if any of the programming stuff really took.  I don't have much of a chance to utilize it in my new job, so I can't confirm or deny if it's a good course. >.> <.<

 

I wish I had started doing the course much earlier in my unemployment, since I could have used that time to play with things a bit more and figure out what I could use it for.  As it is, most of the time I'm just using excel things these days. (Though I admit in one of my better excel sheets I definitely used my Python knowledge to set up a thing - and grumbled the whole time that I could do in two dozen or so lines what I was taking forever in excel...)