Fixing CIFS/Samba Browse Speed on OSX

One thing that has always frustrated me on Mac OS X is the impossible slow directory listing and browsing speeds on CIFS/SMB shares, Apple’s devices, such as the Time Capsule and OS X shares work perfectly, but anything running Samba has this amazingly slow response on any folder with more than 200 files.

Today i’ve been finally configuring my FreeNAS installation on my HP Gen8 Microserver, and after a good twenty or so minutes researching the issue I found a small post on the FreeNAS forums suggesting the following settings:

ea support = no
store dos attributes = no

Boom, quickly added to the configuration files and browsing now flies. Next is to try and improve the overall transfer speed over 25MiB/sec.

Introducing the Home Lab

Driven by some techie mental disability and the thirst to understand more i’ve slowly expanded out my home network into a “homelab”. A few months ago I picked up a cheap HP ProCurve 2824 from eBay, its a great gigabit switch with Layer 2 and basic Layer 3 capabilities, after a quick retrofit of the fans with some nice quiet Sunon Maglevs its been ticking over nicely as the core switch of my network.

In addition to the new switch I “acquired” Jo’s PC to use as a VM host, with some extra network cards and a bit more memory its now serving as a multifunction machine; pfSense, various lab VMs, and monitoring systems.

From time to time i’ll be posting about my latest experiments, what i’m learning, and how its now presenting a even larger drain on the electricity than it was previously.

Resetting the TP-Link TL-SG3210

In the hunt to introduce VLANs across all segments of my home network I managed to pick-up a L2 managed switch for a lot cheaper than I expected, the TL-SG3210 only offers the bare basics but its enough to get some control to the last remnant of the unmanaged network hidden behind a powerline ethernet adapter. At £36 I couldn’t say no to it.

As to expect, it all came pre-configured for the last user’s network, and this time I had no helpful IP sticker like the HP 2824 (I have no idea how I managed to reset that). This switch however had a RJ45 console port, which luckily responded to a standard Cisco rollover cable. Once you’re all plugged all you need to do is:

  • Set the port to 38400 baud, 8 data bits, No parity, 1 stop bit (8/N/1)
  • Reboot the device, when prompted hit CTRL+B
  • At the prompt reset then reset again and the device should reboot.

Once alive again the switch should be available on with admin as the username and admin as the password. I’ve yet to really delve into the CLI of this device but i’d expect some nicer features to be tucked away in there, for the moment its doing a fine job of splitting out that dirty Murdoch device (Sky TV box) and the snooping LG TV off the main network.

Office 2016 for Mac

First of all, this is in no way a product review or anything that should resemble any sort of insight into Office 2016, as the sidebar suggest this is just my inane ramblings on the subject, be it insightful or just utter crap.

For the longest time it seems that the Office team and the Mac team at Microsoft have never really communicated except for throwing the odd document over regarding docx compatibility or some other quirky VBA feature they added in to sooth some crazy corporate. The Mac version of Office has always been the step daughter that the family are pleasant about but don’t really want to talk about them, from the Mac user perspective they’ve always come out with a workable product but it had its odd quirks and never really fitted in with the Mac OS UI/UX style guidelines.

It seems this has changed, as many people have commented in various blog posts over the last year or two that Microsoft is changing, faced with several big companies slowly eating their lunch while they’re not looking it seems that the new batch of products being thrown out are aimed to impress, and Office 2016 for Mac is one of these.

I mean, look at it.

So OK, its not all about the GUI fluff, what really counts is the UX and how everything works together. I feel confident to say you could put someone who is a heavy Office for Windows user in front of this version and they can use it without any major issues. What bugged me in the past is that Office for Mac had its own style and layout which people had to get used to, it seems that the Office and Mac teams are actually working together now and the result is a great product.

I’m sure that after a few weeks usage i’ll be at my usual situation of wanting to pull my hair out trying to get something done, but for the moment i’m basking in the UI goodness…

DVB-T and SDR with the RTL2632

I spotted a Youtube video the other day that talked quickly about SDRs (Software Defined Radios) and how you can pick up one for $20, which is a massive difference from the £300-400 devices I spotted a few years ago. Of course, I decided there and then that i’d grab one to experiment with and searched Google for the mystical device. As it turns out its the Realtek RT26832 based devices which allow SDR type functionality, and while a lot of devices out there are higher than the magical $20 due to them being advertised as a SDR it was quite easy to find one of these generic DVB-T tuners with the right chipset on Amazon for a grand total of £9. With the order being eligible under Amazon Prime I ended up ordering the item yesterday (Saturday) and it was delivered today (Sunday).

So, straight out of the box and into my Debian Jessie test system and everything worked, no tweaking or hassling, within seconds I had a working DVB adapter and I used the standard DVB tools to scan and create a channels.conf within a minute or so. My last experience with the LinuxDVB stack was around 2005-2006ish with MythTV, the drivers “sort of” worked and everything was a little rough round the edges, it seems the last 10 years have really cleaned up the stack. With that in mind its not really worth posting about getting the DVB-T tuner to work, because it just did…

SDR required a little extra work. I’ve not spent a large amount on time trying to get the full toolset to work on Debian, the rtl-sdr toolset is available as a package with Jessie and can be easily install, the biggest problem was that because I was using my test system I didn’t have a X session running to run anything on. I got everything installed and spun up a rtl_tcp instance without much incident, the biggest roadblock was that you can’t have the DVB kmod inserted at the same time as using the rtl-sdr package tools, but a quick rmmod and blacklist sorted that out, the tools are very quick to point out exactly what needs to be done.

Instead of working on Linux I got everything up and running on my Macbook Pro running OS X Yosemite, while OSX doesn’t have the full suite of tools available a few good ones have been developed for the platform. I found that CubicSDR was by far the easiest to get rolling with, no messing with MacPorts or any other third party packaging tools, just a DMG and a pre-packaged application. While it isn’t as feature complete as some of the other packages out there it does cover the basics to go poking around. Their todo list does looks interesting, especially with the target of having digital demodulation built in.

Quick overview done and i’m now looking for a better antenna. While not being used as a SDR the stick itself will be happily serving as a DVB-T source for my Plex system using TVHeadEnd and with a quick MCx to Coax adapter you can have it plumbed into the household aerial without much issue.

[Update - 201508/31]

Regarding the links earlier on in this post, it turns out that the two tuners are actually different, with the Nooelec it includes an improved tuner chip (the R820T2). As it turns out it is worth investing the few pounds more for this version as its more sensitive and also includes a more stable crystal that won’t require much adjustment.

Heroku and NextAction

A while ago it was announced that Heroku would be changing its price structure, after a few minutes with a calculator I worked out that it’ll be essentially better for my bigger apps, and not so much for the small stuff I run. At the time I totally forgot about the NextAction app i’ve been running for a very long time to manage my Todoist instance. For what the application does and how it runs its really not worth the $7 to host on a full Heroku instance…

So with this in mind I put a few hours into the tool today, it now has a proper and a basic CLI interface, just enough to get it running on my VM host without much issue or changes.

If you’re interested in my branch of the tool, check it out on Github.

Starting Again

This is possibly the 5th iteration of a personal blog i’ve had over the years, I think I started around 2001-ish with a b2/cafelog setup, the software that eventually gave birth to Wordpress. I’m not really much of a blogger and my total number of posts over the years probably total up around 500-600, the vast majority of which have been lost forever between the several system moves and lapse of hosting.

So, what now?

I’ve been a big fan of static generators, I’ve never really felt the need to host a heavy solution like Wordpress and after helping out a few sites in their early days struggling with load i’d rather wish to avoid doing that all over again. Jeykll and Pelican seem to be the “big boys” in the static generator scene, but a new startup of Hugo wrote in Go seems to have that right balance of simplicity and features. In terms of hosting i’ve actually cheaped out and took advantage of Github’s static page hosting, with a simple CNAME I now have a fully featured website with just the bits I need.

For those interested, my site’s code is up on Github. Heres to another few years of idle blogging.

Service Shutdown

As some of you may know, i’m getting married, and with that comes the large cost of the actual wedding day itself. With the price creeping up with every wedding fair we go to (it seems, bitches love chocolate fountains) so I have a mighty need to save as much money as possible. With that I can’t really warrant the £70/month hosting bill i’m having to pay out for various freebie services I offer out to my fellow TEST members and other stuff in general. So the following services will be shutdown:

  • EVEThing
  • Testcursions
  • Scantool /
  • Meowbet
  • Dropbot
  • Queuebot

For the apps that have been custom build for EVE stuff, i’ll organise open sourcing them in a few weeks.

The Grand Evernote Reorganisation

I’ve been an Evernote user for a few years now, and its really changed how I keep track of “stuff” I need to keep stored for the future, be it scanned copies of my rental contract for my house or that 5 line how-to guide I found to fix an annoying issue I have. I don’t think I could last a week without checking in with the reference i’ve built up over time.

I’m by no means a heavy user, my Evernote store consists of around 350 well curated notes, most of which have a large PDF scan or image attached. My notebook collection has evolved naturally with my usage and its ended up with around 50 notebooks for my very meagre number of notes, so it was time to simplify.

The last day or so i’ve been reading a few blog posts from various Evernote users, from the heavy hitters down to the people who use it as a scratch pad for ideas. What I found is that a lot of people seemed to share the same consistent view; split down the important bits into notebooks, the rest can live in one “filing cabinet” size notebook as long as they’re well tagged. Jamie Rubin documented his notebook reorganisation and his method really made sense to me. After around of poking around I had the following layout:

  • Personal
    • Medical
    • Banking
    • Documents
  • Professional
    • Certifications
    • HR
    • Job 1
    • Freelance
    • Payslips
    • P60s
  • Reference
    • Contracts
    • Receipts & Warranties
    • Repairs & Service
    • Filing Cabinet
  • Travel
    • Europe
    • North America
  • Shared
    • Public Notes

I still have some crossover between the notebooks, but its a lot better than the previous 50, i’ll have to let this bed in for the next few weeks to see how this works, but i’m already experiencing the advantages of having a single “Filing Cabinet” notebook with all my miscellaneous clippings in. I’m sure in the near future i’ll have to start mass tagging items instead of depending on flicking through the list of notes, but thats for another post…

Flask, EVE, and no persistence

Recently, another EVE related web-app idea popped into my mind and due to the generally low impact nature of the application I didn’t require a backend data store. For a long time i’ve used Django for nearly anything and everything due to the batteries included nature of the framework, but with this application I could throw it all away and start working with Flask; something i’ve been meaning to get my teeth into properly since I started my large Django based projects.

My tool is already a solved problem, but as is the way of development and EVE i’ve set re-inventing the wheel for the sake of “security” and “counter-intelligence”, well, I spin it up that way but really I just wanted to try and do it for myself. In the last few years EVE has had a small UI overhaul which now allows almost anything to be copied and pasted outside of the game, the bonus is that once inaccessible scans, inventory lists, and channel member lists are now sources of information to be parsed and worked with. A common tool to come out of all this is a “D-Scan” tool that allows quick parsing and overview of the scan results from your directional scanner, over the last few years a good scan parser has become an essential tool of any FC and scout.

In my app i’m taking a new twist on the tool, trying out a few new views and consolidating some of the loved features from other tools into one that I can use. In the process of developing this i’ve set myself a goal of not having this tool depend on a database in anyway, instead using Redis as a caching backend for the various APIs and data stores needed.

The first big problem you need to work with is the EVE SDE (Static Data Extract) and the “Inventory Types”, this table of around 50,000 rows is something the tool will need to categorize the scan correctly. The positive here is that the SDE doesn’t update that often, only when we see content releases will the SDE be updated by CCP and even then the world isn’t going to end by not having the latest and greatest SDE to work with. So my solution was to have a package data file populated with a JSON extract of the data I need and when the data is needed its loaded into memory, the relative memory increase of 1-2mb of RAM is nothing in the overall scheme of the application.

So what about the actual scans and results? Parsing the d-scan data is relatively quick, as its essentially a tab delimited file of a fixed format, combined with a few quick lookups of reference data which is all held in a dictionary in memory makes even a taxing Jita d-scan get processed in a few milliseconds without any major optimization. Once the initial parse is done then the results are dumped to JSON, compressed with zlib, then dumped to a unique key in Redis with a expiry of an hour. The view to show the scan results does nothing more than to take the key from the URL and attempt to grab the results from Redis, decompress, and pass the resulting parsed JSON to the template.

The deployment target is Heroku, and ideally the Heroku free tier, so this has dictated some of the design, for example the zlib compression of the resulting scan is there to shave off as many bytes as possible to get the maximum use out of the Redis 25mb services you have available, with the requests we’re CPU rich but storage poor, so the trade off works quite well in this case. So how would this work in a DoS issue? If one person keeps spamming large d-scans into the system would the Redis server fill up and stop working for all? Well, no, as the config will be set to expire the oldest keys in the case of low memory which would work perfectly for our tool.