Within the overall IT community as well as GIS community there are a number of pushes for “inclusion”.  Increasing the number of women, LGBTQ, minorities (both racial and religious) at these events so they can learn, network and help the diversity within our fields and industries grow.  This blog post is not intended to belittle these initiatives nor comment on the progress that is being made.    With that being said, lets hit the meat.

Its no secret that I’m a single father, and I have moved to a place that is isolated from the larger parts of my community.  As such, when I heard FOSS4G-NA was going to be in Raleigh, I made plans to go.  The biggest issue was my daughter.  She took the week off school, and we loaded up the Edge and drove up from Fort Myers.  I had an artcamp lined up and a baby sitter for a couple of the nights so I could continue to network and enjoy more of the conference.

OR how this should read is

I’m lucky enough to have a job where I can throw money at a huge hurdle for me to come to an “inclusive” conference.

Within the US alone, there are 25 million children (around 35%) are living in single parent households.  Of these, there are about 12 Million single parents (80% female and 20% male).  So, we’re talking about 3% if the overall population.  Which doesn’t seem like a bunch, but its the same size as ALL the non-christian faith’s put together (Jews, Muslims, Hindus, Buddhists).

Few single parents are a lucky as I am, they juggle work and kids, and their work isn’t flexible. Most of the people i know who I told I was bringing my kid along said “Oh thats Tough” or just “Wow” — I KNOW RIGHT.

This shouldn’t be an issue, we have a well developed child care system in the US and I’m sure there are plenty of providers out there, or hell, LOCAL GEOGRAPHY TEACHERS, who might like to make a couple of extra bucks and during a poster session or a planned networking meet up, use one of the breakout rooms to teach the kids a thing or two, or just let them watch some general milktoast film.  Just to allow the single parent 2 hours to network and talk with peers.  But its overlooked, discarded — and why is that….

Because most single parents don’t come with their kids so their voice isn’t heard.  Because there are these barriers to entry that are, frankly, too great.  And honestly, if I was STARVING for interaction with my community, I would have blow this conference off as well.  “Man I’d love to go to that and present all the cool stuff I’m doing, but what about the kid.”

This is a problem that we already have the pieces to solve.

1970: “Why is that woman at this conference”

1990: “Ugh, there is that gay, they shouldn’t come to this event.”

2016 OH at FOSS4G: “Who would bring their kid to this.”

So, to wrap up this entry, I didn’t feel included.  I felt like I could attend part of the conference, but not the whole thing.  Those 6 hours a day I could get to, I did like, but until something changes, I’m pretty sure this is going to be my last FOSS4G.

Also,

Let me give MASSIVE props to the ESRI UC Team for their Map Camp and all the family friendly events they have in San Diego.  Way to go guys, you nail it.

Also, big ups to the people who did interact with my daughter.  You cats are stellar.

I don’t talk about my personal life here very much, as that’s why we have Facebook, and its not the purpose of this blog.  Unless of course those two streams cross in a meaningful way.    Point blank, I’m a single parent with a limited support structure and next to no ex spousal support.  As such, my kid and I tend to be a package deal.

Couple this with my current position where I am doing really really cool stuff and want to give presentations at conferences about it.  Or you know, just GO to a conference and not have to jump through a thousand hoops to figure out where, I can stash my kid locally for a day or two while I learn and network.

At State of the Map in 2014, they had onsite daycare, for free.  Now, I’m not asking this for every single conference to provide free daycare, I’m willing to pay for it.  Like for FOSS4GNA this year.  I was able to swing putting my kid in a local artcamp for the time of the conference.   If conference organizers create a relationship with a local provider  or camp that’d be enough.   (If you want to know about the artcamp, ping me)

This is the one place I do have to give credit where credit is due.  The ESRI International User’s Conference and their Map Camp does a stellar job at supporting this community.

So while its nice to think about inclusion of groups, lets not leave out the single parents/parents who don’t want to be away from their kids.  Providing child care facilities, not on site, but nearby and making it logistically easy for parents to do this  will help their decision to spend their training dollars not on some local class, but to attend your conference instead.

Like any blogger, I have drafts of posts that just aren’t jelling, not saying what I really want to say, ideas that require a bunch more time to sort out, or they’ll just get me in trouble.  With the new year around the corner, I’m going to dump the titles here, and brief synopsis of what they were about.  Cleaning off the plate for 2016.

1. Title – “GSA: Go Slow Always”

This entry detailed, too much so, my time while working as a contractor over at GSA. I wrote and researched RFPs and RFIs, and worked with the Cloud group to approve a large Cloud implementation to be “Fedramp” approved.  I ran this one past my lawyer, and she told me to hold off. Basically, GSA has no fucking clue what they are talking about, and the large contractors drive the process. Oh,and the sweetheart 18F, its lipstick on a fucking pig. The old guard won’t release their power to them. “The processes are here for a reason.” Yeah you worthless tax leach, so you can go on junkets to where ever large contractor is having a “symposium” on Emails as a Service. Seriously, never look under the hood at GSA, it will turn you Libertarian.

It never jelled because I could never maneuver around the guidelines given me.

2. Title – “Soulsucking Contractors NOT to work for”

When I found out I was going private sector, I started to pen this as a giant F you to Federal Contracting in general. But cooler heads, read not mine, talked me out of it. I am going to say this, never work for the Department of State. Their processes and structure is based on some 1970s concepts titles and lines of communication. Worse Agency Ever.

It never jelled because I’ll write a tell all after I retire.

3. Title – “Dealing with the Social Media Deluge of the ESRI UC.”

Basically, this one deals with how not to unfriend all your Kool-aid slurping, push button analyst friends on social media during the UC. I was trying to work some high school analogy in here, about punk rock and the anti Prom stuff. It either came across as douche or elitist or just plain old mean.

It needs work, and maybe I’ll wordsmith it better for this years.

4. Title – “Command Line Spatial”

I have this theory where ideas travel around the globe and smack liked mined people at the same time. The week I started working on this one, both Fee and Dollins posted about it. So I dropped it. No one likes a bandwagon.

5. Title – “So this is what its like to be Clark Kent”

By far my most douchy post, and there wasn’t anyway to make my point without basically calling everyone at the SWFL GIS Symposium “A bunch of damned idiots.” Basically, I can’t believe how little Spatial Professionals know about their own industry.

There are others, but these are the most developed.

When its all said and done 2015 was a turbulent year for me.  On the other side of all that crap, I’m glad I landed where I did.

I’ve been remiss with this blog for the past month or so.  My brain is wrapped around this one post, and its just not jelling, but thats a discussion for another day.

With the end of the year rolling around its time to tell yourself that you’re going to lose weight, get the gym more, and spend more time with your kids.  But lets face it, food is amazing, you’re fucking lazy and the kids are great for MAYBE an hour.  Why not make a resolution that will enhance your career.  Or, well while we’re being honest, won’t send you to the corner office , but will at least give you a different perspective.  At the very least, I’ll make more sense on twitter.

This is for all Spatial professionals, from Analysts, to Developers to whatever the hell a Specialist is.

1. Resolve to mess around with Raster Data:

When I first started in this industry, Raster data wasn’t cheap nor accessible. It was big, expensive, and the processing power of systems at the time (IT HAS 4 GIG OF RAM), Raster was hard to deal with. Now between Amazon hosting Landsat, Drones, Planetlabs and just people tying iPhones to Weather balloons, Raster is becoming ubiquitous. Thanks Google. So download some, find a tutorial and figure things out. Then, find some more, and solve a silly problem you make up, like “How many blue cars are parked on the street near my favorite restaurant while this data collector was above. Else, you could buy a book or find a lab.

2. Resolve to go hang out with other Spatial Folk:

Some of us work in a geo vacuum. We’re stuck with IT nerds, Environmental Geeks, or at worse, Douchebag Intel Analyst. YOU NEED TO GET YOUR MOTHERFUCKING GEO ON. Between Maptime Chapters, ESRI Regional, National and International events, FOSS4G Conferences or just use Meetup and type in GIS.

You’ll meet like minded individuals, and maybe someone you follow on the Twitters, or a blog you read. Swap knowledge, ideas, methodologies and spit if you’re single. The key thing to remember is, everyone you meet knows something you don’t.

3. Resolve to leave your comfort zone:

Get to work, start application X, make sure Y is up and running, same requests same methods. Humans love repetitive task; keeps the mind sharp. NOOOO. Leave you comfort zone. Thinking about your processes, is there a technological or new methodology that might make the process faster, simpler, or just better? Step away from what you’ve done, and walk toward what you are going to do. Are you an ArcSDE Database expert? Learn PostGIS, Oracle Spatial, or even that train wreck of SQL Server Spatial. Desktop Specialist, pick up Python, or recreate you process in QGIS, just to see how it does it.

4. Resolve to flirt a bit with Data Science:

Is no secret that I believe Spatial is going to be consumed by Data Science. Now, I’m not suggesting you run out and learn R or Pandas. Just read enough to gain an understanding of what a Data Scientist does, and how they visualize data. There are Podcasts, Websites, dead trees and even hot Expats with blogs.

5. Resolve to create something, not for work but for fun:

Start a blog, make a pretty map and submit it to the 2017 Geohipster Calendar. Start a podcast. Yes its hard, yes its hard to eeek out an extra 20 or 30 minutes a day, but start something. Stop being just a consumer, and be a producer. Or as Kevin Smith would say “Be a maker, not just a taker.”

Just pick one, and take a little bit of time to do it. Overtime it will become part of you and your career make up. Trust me, you’ll be a better person, employee and geographer.

So, its been a little over two months since I’ve moved down to FL from DC.  While there are a number of positive effects – spending more time with my daughter, having a job that rewards my creativity and ideas, to missing my friends in DC, and not having a large social network down here.  I’m working on it, but things like this take time.

However, I do have a recruiter who was DETERMINED to get me to move back.  The position was one of my favorites, but its morphed since I had it.  When I started on the project back in 09, GIS wasn’t the focus, and we had spatial data in non spatial databases, so I worked to make sure we never touched ArcGIS Server or any licensed ESRI product if we could help it.  Hell, I even slide openlayers into the source code.  The idea was to do it all in javascript, before pressure from both button pushers within the organization and ESRI themselves applied pressure to “Retrofit the VIPER flex viewer”  I tossed the towel in at this point, and went on to other things. I revisited the project a couple of times, and you could tell that it was just a “configure and deliver” project.  I ain’t got time for that.

Anyway, the position was up for recompete, and this little firm, who subbed to Booz Allen, won that particular seat.  When my resume came up in this recruiter’s results, I guess he became determined to “land me.”  Anyway, after back and forth with this joker, he finally said “What will it take to get you to move back.”

So  I sent him a list.

• Relocation Expenses
• Mid 5 figure hiring bonus – “This would ease the pain from the crap I would get from people”
• Salary, high 200s
• A 3bdrm, 2 full bath house or condo, in an area of DC of my choosing,  owned free and clear by me.  (I’d pick up the property taxes, I’m not a monster)

Needless to say he hasn’t returned my email.  But in hindsight, I might not even take this offer.  What, so I could go back to being a GIS version of a SharePoint Administrator.  Before I left I looked hard at the jobs in DC, none of them were really dynamic or “fun.”  It was all, wire this crap up then sit on it til the contract is over.  Having a dynamic position, where I’m in control of what happens within the sandbox is worth a bunch.  I mean, I have no intention of staying here forever, but right now I like it, and I’m happy.  Those two things trump all the beltway bragging rights and being able to say “Oh no, I ACTUALLY live in DC, not VA.”  snobbery.

Government Contracting and DC in general stopped being for me.  I was there for 20 years, and I wouldn’t trade those times or experiences for anything.  I’ll visit, and I might return someday, but not in the near future.  Life is different on the edge of civilization, frustrating at times, but good.

I’m getting my feet under me, and I have a few technical blog posts outlined – as such I’ll be back on the blogging pony soon.

I’m still getting my swamp legs under me.  Sorry for the delay,  but moving is hard.  Also, who has time to blog when you live in a place with airboats.

Between wrapping up my work duties at my current job.  Packing for a move to SW Florida, and prepping for my next job, I’m uber swamped.  No pun intended.

I should be back in a couple of weeks, but right now TOO MUCH GOING ON.

This is the post this week folks.  I know you are all underwater keeping up with the happenings out of the UC, whatever they may be.  Below is a animated gif of a dog on a wave runner….Enjoy

My kid came back after being with her mom for a couple of weeks, so I’ve been spending time with  her, and unplugging as much as I can while still working.

Really, I just wanted to drop by and wish those going to the ESRI UC a good trip.  I went for 10 years straight, and just got bored with it.   I talked about that here, last year.  I might go again soon, but I really prefer to spend my capital on FOSS4G.

Anyway, here is a bulleted list of things you should do while in San Diego:

• Skip the main hall for the plenary, head up to the social media lounge and lay down on a comfy couch and watch it live
• Ask to sit with someone who is eating alone.  You have something in common, and you’ll never know who you will meet.
• Crash a Big Defense or Intel Party.  One time a buddy and I got in using other people’s business cards, we got bonus drink tickets.
• Blame the shitty wifi for not being able to do work – trust me
• Make at least 2 new friends.  Not twitter followers, not network nodes – actual friends
• If more than 2 people in you group suggest going to Tijuana, GO
• Stay “on task” with what you want to go to.  You’ll want to go to almost everything you see.  Put together a thoughtful plan and stick with it, but make sure you have backup sessions.  ESRI does not provide any QA/QC for these sessions and they might suck.
• Don’t be afraid to walk out of sessions.  No one’s feelings will be hurt
• Don’t be afraid to just hang out – remember most of this stuff will be blogged/tweeted about, sometimes the connections you make are a better use of time
• Follow twitter hashes, and don’t be afraid to say stuff like “Where is everyone partying tonight.”
• If you find someone lost from Comicon help them out
• You are all geographers, and maybe responsible adults.  If you see anyone in trouble, passed out, having an issue with the staff, whatever.  Help them to a cab or uber, or see if you can help in anyway.  GIS is a supportive community, help a GeoGeek out.
• Last, but not least.  If you are planning on going to the Thursday party, and your flight is Friday.  After you last session on Thursday, return to your room, PACK, then head out.  Trust me on this one.

Everyone be safe, and somewhat sane.

Anyone worth their salt in the Geoverse has complained about the Shapefile.  Its limits, its size.  The fact its not a file, but a collection of files, hence why I and others call it the Shapefolder.  But lets be honest – our beautiful datasets being analyzed in Q, or pushing out a REST service, or GeoJSON or whatever you want from PostGIS, has spent at least a portion of their life as a shapefile.  In an un-scientific survey, 178% of all geodata has spent time as a shapefile.  The reason its over 100% is because the files were bloated before, because…shapefolder.

No, I am not here to sing the praises of the Shapefolder, much like iceberg lettuce, dating and that film on the top of yogurt, you have to deal with this crap to get to the good stuff – the data they contain. This post is about how to crack open a Shapefolder, and get to that creamy data goodness, without using a desktop tool.  Because sometimes you just want the data for analysis.

We’re going to discuss, R, Python, Julia and Java(badly), and how they can consume, and analyze the data in a Shapefolder  – adding to their data to your analysis.  There are other methods to do this, such as Spotfire’s native Shapefolder importer, but I wanted to talk about you just banging it out in front of an IDE, or notepad.  This isn’t a deep dive, just a skim over what is possible.  We’ll get into visualization next week – maybe.

First some caveats about Shapefolders.  First, they are limited to around 2.2gb of storage.  While this does seem like a sea of data to a normal human, that’s only about 70 million records.    Second, if you shapefolder has ever touched ArcGIS, it will replace all null values with 0.  ArcGIS does not support NULLS in shapefolders, so you may have some data cleaning in front of you OR, ifin your stuff 0 is basically null.  If you are going to be creating shapefolders from data, then there is a whole slue of things you’re going to need to be concerned about, field size, rounding errors, or taking your dead dog to a pet semetery, only to have it come back a possessed killing machine – or was that the plot of a Steven King novel.  Either Shapefolder or Steven King novel, we’re about to open up “the horror.”

<Edit>Some Snowback called me out for not speaking about OGR.  This post assumes there is data in a shapefolder that you want to help perform further analysis, so you want it integrated into your model/language.  Not just pulling the data out for the data </Edit>

PYTHON

Python is a quick clean and has a number of robust libraries for Spatial, and its the most popular language for Data Science and GIS.  I prefer PySAL, it seems to be the most complete of all the spatial packages for Python.  Not only does it allow you to crack open a .shp, but you can also just crack open the .dbf and harvest the data from there.  Which does speed up the data ingestion process if you aren’t doing a visual.

If you know you’re just going to be dealing with Shapefolders, PySHP is another library you can check out.  While it does not offer the complete geospatial analytics package as PySAL, if you’re just cracking open shapefolders.  One of the advantages of PySHP over PySAL, is that it will crack open any of the the Shapefolder’s file types, so if you’re just grabbing projections, then you can import and harvest that file.

Its also feasible just to use NumPY, but you’ll be writing a bunch of code, and none of us want to do that.

R

Like Python, R is a language well suited for Data Science and statistics, mainly because it was created by Statisticians.   There are a whole SLUE of  Spatial Libraries for R, the one I am partial to is SpatStat.  SpatStat allows you to do, well what everthing else does so far, crack open a shapefile, and get to that data without using a desktop application.  As with all things R, there is a mailing list.  If you only have two take aways from this blog entry, its that if your shapefile has ever touched ArcGIS you’re going to spend hours cleaning it, and that R users love mailing lists.

Julia

Julia is a recent interest of mine.  I build an R model that needed to scale, and you know what.  R scales….poorly.  Nosing around a bit and I found Julia.  We’ve only been dating for a couple of months, but as far as I can tell, this relationship is going to last a few years.  <– that sentence is why you should never name a language after a woman.

Regardless, Julia has a library just for parsing out shapefolders, strangely named Shapefile.  That being said, Julia can also call functions from other lingos.  So, if you wanted to you could bring PySAL, or if you wanted to pull async data,  you could hook her up to Node, pulling in data and then munching it using PySAL or SpatStat or whatever Java package does spatial stuff.  So Julia is really the dark matter of the Data Science world at this point.

Java

I’m not a Java guy, never really have been – that being said there is a really robust library for Java from the OSGEO Folks, GEOTools.  I’ve never used it, but its been around forever and has a strong user community and support.  But yea, go there, download it play with it.

Next week we’ll talk about Visualizing geospatial data without Arc or Q, and lead into  a deeper discussion as to “Why do I need these things in the first place?”