For years, I've had an on-again off-again relationship with Evernote. As someone who is terminally unorganized (my doctor says I only have 40-50 years left to live because of it), my biggest problem was finding a way to make all of that data ubiquitous so I can manage it easily and everywhere. I'm back on again with Evernote, primarily due to the introduction of their Chrome extension, Clearly.
For some time, I'd used Readability as a way to get the data I wanted into whatever I was using as a note-storage vault. Using the Readability bookmarkelt, I could easily trim pages from ad-flashing, comment-choked nightmares into slim documents containing only the information I cared about. Someone had even combined the Evernote Clipper bookmarklet with the Readability one making for a one-click solution. But, it never really worked perfectly and, as stated before, Evernote never had enough love from me to make it long term.
But, Clearly DOES work extremely well! And because of it, I've taken another hard look at Evernote and have fallen in love with it. I'm inside this tool every day now, sometimes 5-6x each day...either updating my journal or trying to get the hundreds of notes I already have squirreled away into a neatly organized mess.
As with most tools, once you start power-using it, you start to see the potential to use it to solve other, apparently unrelated issues. Such is what lead to this idea I'm about to outline...
As with most geeks living in the 21st century, I suffer from information overload. I'm subscribed to about 50 different sites via their RSS feeds and use Google Reader to keep up with it all. But, I've found the way I do things to be too slow and cumbersome. My primary issue is with Gawker sites like Lifehacker and IO9, the two sites I feel compelled to visit every day. I have no idea what the issue is with these two sites, but I can have Chrome open all day long, with dozens of tabs generally in view at any time. The instant I open more than two pages from Gawker, my whole machine crawls to a halt.
Now, it's very likely it's some combination of my extensions and the proxy at work as I don't experience this issue too often elsewhere. But, seeing as most of the visiting of these sites occurs when I'm in between tasks at work, it's a pain in the ass I had to find a solution for. I also had the problem that it might not be something appropriate for me to read at work, so I'd fill up my inbox with things to read later...and then never get to them.
Additionally, if I find an article I like, I'm likely to want to save it. Since coming back to Evernote full time, I've been using Clearly to snip them up. If what I want is going to end up in Evernote anyway, it seemed to make some kind of sense to just figure out a way to eliminate the middle man.
And that's where If This Then That comes in. IFTTT is a site that will automate your social media for you. For example, I've got a task setup that will fire every time I post a status message on Facebook and send it to me as an e-mail. Since Facebook's search leaves a lot to be desired, plus I wanted to have an archive of things I posted there, this process makes my social network history much more manageable. (Recently, I've started switching these tasks over to put the items in Evernote instead of sending an e-mail.)
The secret sauce in this task is IFTTT's ability to react to an item being starred in Reader. When I do that, IFTTT posts that article into an Evernote notebook named RIL (for Read It Later, see what I did there?). I can then go in at my convenience and just read the articles in Evernote. As an added plus, they're already in Evernote, I just have to drag articles I want to keep into the appropriate notebook when I'm done reading. Or, if I don't want to keep it, I just delete.
One last tidbit: most sites will only post a summary of an article in RSS. Some, like Lifehacker, will offer the full articles on a separate feed. If they don't, you can try WizardRSS. It's a site that will pull full articles instead of summaries from the site and give them as their own RSS feed. It doesn't work on every site, but it gets enough of them that I watch each day.
I've been using this method for about a week and a half now, and I have to tell you it's made my RSS surfing so much more efficient. For example, when I'm done looking through a particular site's articles, I always mark the whole site as read so I easily know what's got new stories when I come back. The problem is, if I found an article that I had the slightest interest in but didn't want to read it at that time, I'd just leave it unread in Reader. Messy, messy! No more, I've moved all of those waiting to be read articles into Evernote and I'll get to them when I get to them.
Now, Reader is generally empty except for the few new items. I can scan through all of my feeds in a matter of minutes and star just the items I MIGHT want to read. I can then read them on my phone or computer at my convenience. I gotta tell you...it's made me so efficient at getting through my news I run out everyday and that's something that's NEVER happened before!
Showing posts with label Computers. Show all posts
Showing posts with label Computers. Show all posts
Thursday, December 15, 2011
Tuesday, April 26, 2011
Resizing Firefox
In my last post, I talked a bit about my migration to Firefox 4 and some of the extensions that generally follow along with me. Today, I want to discuss some of the tweaks I do to the interface to make it more usable.
On of the changes to the technology field over the last couple of years has been the migration of computer monitors to widescreen format. While not ideally suited to use in this manner, there's a very simple reason we're forced to have them: HDTV. As the television and film industries move to widescreen, manufacturing of monitors has had to follow as a matter of course. Or, cost, really. The same companies that make TV screens also make computer monitors and it costs money to build LCD/LED/Plasma screens in a variety of formats. The fewer formats manufacturers have to deal with, the lower costs end up being.
The fact that they're not really what consumers want really doesn't come into play. If they're the only option available, then you'll buy them. And, so, we're stuck trying to scroll around the screen to read the entirety of a webpage or Word document that's formatted for a taller screen. Such is life, this is where we are, so let's make the most of it.
A lot of sites have been sizing up the latest offerings from the browser producers and judging their screen real estate usage and evaluating them primarily on their vertical usage of space. If I recall correctly, in the latest offerings, Firefox 4 is the current winner. It apparently undercuts Google Chrome, the former champ, by some 3 or 4 pixels! Whew! I can almost read the top half of one extra sentence with that!
While it's nice to see that Google, Mozilla and Microsoft are all taking into account screen ratios now, it's always been kind of a moot point for me. After all, I've been taking it into account since I got my first widescreen a few years ago and came up with my own solutions by tweaking Firefox. The solutions below are not possible with Chrome or IE because their layout is pre-determined. Firefox is the only browser that gives me the flexibility I need to size things properly.
This is what my browser looks like:
The Basics
For some reason, there are folks out there who love Firefox, but don't want to add extensions to it. I can grok wanting the absolute fastest browsing experience you can try to muster, but I just think you're missing out on so much! Oh, well, this section can be for them.
The first thing you'll notice in my screenshot above is I've reduced the number of toolbars on screen to just one. This is easy enough to do, simply right-click on one of the buttons on your toolbar (such as the big Back one) and choose Customize. You can drag bits of the interface around, as well as remove some. The first thing I do is get rid of the Search box and Back/Forward buttons.
Losing the Search box isn't a big deal if you use Smart Keywords to search from the address bar. I've got a few of these setup so, for example, if I want to do a Google search, I type "g" in the address bar and follow it with my query. Similarly, I have one setup for Google Image search, Bing, Dictionary.com, Wikipedia and a few others. I also have one for Google Maps to get directions easier. With these, I really have no reason at all to have a Search box taking up space.
Not having Back/Forward buttons might seem a chore...unless you read about FF Rocker in my last post. :) The goal is to get rid of items on screen that can be actioned using some other method..
Once I've subtracted, simply move the Address Bar up into the menu Toolbar followed by the widget named Bookmarks Toolbar Items. Things might be a bit cramped at this point, so you might need to make some additional changes. By default, the Address Bar is a bit wide for this setup. So, we need to find your Firefox profile. This is the directory that contains all of your personal settings for Firefox. This article describes nicely how to locate it. Once you've found it, you're going to want to go into the chrome directory (create it if it doesn't exist) and modify userChrome.css (again, create it if it doesn't exist. userChrome.css is a configuration file for aspects of the interface. I haven't done much twiddling, but two changes I do make:
#urlbar, #urlbar-container {min-width: 500px !important; max-width: 500px !important;}
#star-button {display:none !important;}
The first one sets the minimum and maximum widths for the Address Bar to 500 pixels. For the most part, the only bit of a URL I generally care about is the domain, and that generally fits within those 500 pixels. There is an issue with the new "encrypted page" indicator in later versions of Firefox. This takes up a wide part of the Address Bar to show you the domain the certificate is for. Sometimes, if the domain name is long enough, it'll take up most of the Address Bar, but I've never encountered a time where I thought "damn, my Address Bar just isn't wide enough" at 500 pixels.
The second entry removes the star button from the Address Bar. This button is used to bookmark a page, but I keep my bookmarks in Google Bookmarks, so hardly ever use it. Better to use Ctrl-D to bring up the bookmark tool than waste some space.
Speaking of bookmarks, you'll notice my Bookmarks Toolbar is fairly sparse with just favicons
representing most sites. To get a slim and trim Bookmarks Toolbar, simply go to the properties for each of the bookmarks and remove the Name. Very few sites these days don't use favicons, so you can always identify them without their name next to them.
You might also notice the folders on the Bookmarks Toolbar. I do have a number of sites I hit on a daily basis that don't have RSS feeds. My morning routine consists of hitting the "M" folder (for morning), going to the bottom and hitting the "Open all in tabs" link. This does as it says and all of my morning news is available for me to peruse in just a few moments. In the afternoon, there are some sites I hit that are specific to that timeslot. Hit the "A" folder (guess what that stands for) and do the same. The remainder are either work related or something I like to keep handy, but don't want to waste an icon on the toolbar.
Extending It Further
While the above can go a long way to trimming Firefox down vertically, there are still some cool things you can do with extensions to really bring it down to size. The one item that's taking up the most space at this point is the Tab Bar. Really, why do we need it along the top? Because that's the way we've always done it? This is where Vertical Tabs comes into play. There are a few options in the add-ons mart, but this is the only one that works with FF4. That's okay, though, because I've tried a few of the others and they add a lot more features than I really needed. I wanted my tabs on the side, that's it. With the tabs off to the side, it actually feels a bit more natural. After all, look at Windows Explorer (not Internet Explorer). Are the panes divided up top and bottom? No, they're next to each other. When you go to a site, navigation menus are typically on the side, and so on. It really doesn't take much getting used to once you install it.
Since you've squinched as much as possible into your menu bar, it's probably looking a bit crowded, right? Well, why not trim out some of the things you don't use as often? When's the last time you hit the Help menu? The Edit menu? Okay, some folks might still like that one, but I know how to hit Ctrl-C and Ctrl-V, so very rarely use it. :) Since the only bookmarks I store locally are on the toolbar and I'm using Ctrl-D to bookmark new ones, do I really need the Bookmarks menu item? Nope, so off they go thanks to Menu Editor. Trim your menus down how you like, but I've gotten rid of everything except File, View and Tools. It may seem a radical surgery, but the patient and I have been doing fine for ages now. :)
About the only change left at this point has already been done for you in Firefox 4: getting rid of the Status Bar at the bottom. Most folks seemed to like that change, some didn't. There's an extension called Status-4-Evar to get it back if you really like it. I haven't had much complaint working without it so far.
So, that's it, my slim, trim Firefox 4 setup. On a widescreen monitor, I get as much space devoted to browsing as I possibly can, and didn't lose any functionality. In fact, usage is generally improved some. Got any tweaks you know I might've missed?
On of the changes to the technology field over the last couple of years has been the migration of computer monitors to widescreen format. While not ideally suited to use in this manner, there's a very simple reason we're forced to have them: HDTV. As the television and film industries move to widescreen, manufacturing of monitors has had to follow as a matter of course. Or, cost, really. The same companies that make TV screens also make computer monitors and it costs money to build LCD/LED/Plasma screens in a variety of formats. The fewer formats manufacturers have to deal with, the lower costs end up being.
The fact that they're not really what consumers want really doesn't come into play. If they're the only option available, then you'll buy them. And, so, we're stuck trying to scroll around the screen to read the entirety of a webpage or Word document that's formatted for a taller screen. Such is life, this is where we are, so let's make the most of it.
A lot of sites have been sizing up the latest offerings from the browser producers and judging their screen real estate usage and evaluating them primarily on their vertical usage of space. If I recall correctly, in the latest offerings, Firefox 4 is the current winner. It apparently undercuts Google Chrome, the former champ, by some 3 or 4 pixels! Whew! I can almost read the top half of one extra sentence with that!
While it's nice to see that Google, Mozilla and Microsoft are all taking into account screen ratios now, it's always been kind of a moot point for me. After all, I've been taking it into account since I got my first widescreen a few years ago and came up with my own solutions by tweaking Firefox. The solutions below are not possible with Chrome or IE because their layout is pre-determined. Firefox is the only browser that gives me the flexibility I need to size things properly.
This is what my browser looks like:
The Basics
For some reason, there are folks out there who love Firefox, but don't want to add extensions to it. I can grok wanting the absolute fastest browsing experience you can try to muster, but I just think you're missing out on so much! Oh, well, this section can be for them.
The first thing you'll notice in my screenshot above is I've reduced the number of toolbars on screen to just one. This is easy enough to do, simply right-click on one of the buttons on your toolbar (such as the big Back one) and choose Customize. You can drag bits of the interface around, as well as remove some. The first thing I do is get rid of the Search box and Back/Forward buttons.
Losing the Search box isn't a big deal if you use Smart Keywords to search from the address bar. I've got a few of these setup so, for example, if I want to do a Google search, I type "g" in the address bar and follow it with my query. Similarly, I have one setup for Google Image search, Bing, Dictionary.com, Wikipedia and a few others. I also have one for Google Maps to get directions easier. With these, I really have no reason at all to have a Search box taking up space.
Not having Back/Forward buttons might seem a chore...unless you read about FF Rocker in my last post. :) The goal is to get rid of items on screen that can be actioned using some other method..
Once I've subtracted, simply move the Address Bar up into the menu Toolbar followed by the widget named Bookmarks Toolbar Items. Things might be a bit cramped at this point, so you might need to make some additional changes. By default, the Address Bar is a bit wide for this setup. So, we need to find your Firefox profile. This is the directory that contains all of your personal settings for Firefox. This article describes nicely how to locate it. Once you've found it, you're going to want to go into the chrome directory (create it if it doesn't exist) and modify userChrome.css (again, create it if it doesn't exist. userChrome.css is a configuration file for aspects of the interface. I haven't done much twiddling, but two changes I do make:
#urlbar, #urlbar-container {min-width: 500px !important; max-width: 500px !important;}
#star-button {display:none !important;}
The first one sets the minimum and maximum widths for the Address Bar to 500 pixels. For the most part, the only bit of a URL I generally care about is the domain, and that generally fits within those 500 pixels. There is an issue with the new "encrypted page" indicator in later versions of Firefox. This takes up a wide part of the Address Bar to show you the domain the certificate is for. Sometimes, if the domain name is long enough, it'll take up most of the Address Bar, but I've never encountered a time where I thought "damn, my Address Bar just isn't wide enough" at 500 pixels.
The second entry removes the star button from the Address Bar. This button is used to bookmark a page, but I keep my bookmarks in Google Bookmarks, so hardly ever use it. Better to use Ctrl-D to bring up the bookmark tool than waste some space.
Speaking of bookmarks, you'll notice my Bookmarks Toolbar is fairly sparse with just favicons
representing most sites. To get a slim and trim Bookmarks Toolbar, simply go to the properties for each of the bookmarks and remove the Name. Very few sites these days don't use favicons, so you can always identify them without their name next to them.
You might also notice the folders on the Bookmarks Toolbar. I do have a number of sites I hit on a daily basis that don't have RSS feeds. My morning routine consists of hitting the "M" folder (for morning), going to the bottom and hitting the "Open all in tabs" link. This does as it says and all of my morning news is available for me to peruse in just a few moments. In the afternoon, there are some sites I hit that are specific to that timeslot. Hit the "A" folder (guess what that stands for) and do the same. The remainder are either work related or something I like to keep handy, but don't want to waste an icon on the toolbar.
Extending It Further
While the above can go a long way to trimming Firefox down vertically, there are still some cool things you can do with extensions to really bring it down to size. The one item that's taking up the most space at this point is the Tab Bar. Really, why do we need it along the top? Because that's the way we've always done it? This is where Vertical Tabs comes into play. There are a few options in the add-ons mart, but this is the only one that works with FF4. That's okay, though, because I've tried a few of the others and they add a lot more features than I really needed. I wanted my tabs on the side, that's it. With the tabs off to the side, it actually feels a bit more natural. After all, look at Windows Explorer (not Internet Explorer). Are the panes divided up top and bottom? No, they're next to each other. When you go to a site, navigation menus are typically on the side, and so on. It really doesn't take much getting used to once you install it.
Since you've squinched as much as possible into your menu bar, it's probably looking a bit crowded, right? Well, why not trim out some of the things you don't use as often? When's the last time you hit the Help menu? The Edit menu? Okay, some folks might still like that one, but I know how to hit Ctrl-C and Ctrl-V, so very rarely use it. :) Since the only bookmarks I store locally are on the toolbar and I'm using Ctrl-D to bookmark new ones, do I really need the Bookmarks menu item? Nope, so off they go thanks to Menu Editor. Trim your menus down how you like, but I've gotten rid of everything except File, View and Tools. It may seem a radical surgery, but the patient and I have been doing fine for ages now. :)
About the only change left at this point has already been done for you in Firefox 4: getting rid of the Status Bar at the bottom. Most folks seemed to like that change, some didn't. There's an extension called Status-4-Evar to get it back if you really like it. I haven't had much complaint working without it so far.
So, that's it, my slim, trim Firefox 4 setup. On a widescreen monitor, I get as much space devoted to browsing as I possibly can, and didn't lose any functionality. In fact, usage is generally improved some. Got any tweaks you know I might've missed?
Friday, March 25, 2011
A new day, a new Firefox
Firefox 4 was released to much fanfare the other day, and I decided to give it a shot. Since I only use the Portable versions of Firefox (more on that later), it's an easy enough thing to give new versions a try and if I don't like them I can wait until they get better. So, I hit the Portable Apps site and sure enough there was the release version of FF4.
On first launch, performance seemed to be noticeably faster than 3.6.13, the previous version I was using. Opening Gmail was near-instantaneous and other sites just popped into view as soon as I finished typing the URL. It only took a few minutes before I decided this was worth checking out further, so on to the next most important step: are my extensions available?
One of the main reasons I use Firefox is the customization through extensions. If the extension author hasn't updated it, that usually restricts if I'm going to move to a new version of the browser or not. This isn't true of all of them, but I do have a suite of "core" extensions I require at all times:
The "Musts"
Adblock Plus
I'm an American. I want everything, and I want it for free, dammit! :) I get it, sites use ads to generate revenue. Some generate a lot of revenue from them, but those sites typically have so many ads on them their content is almost unusable. Adblock Plus does a great job of preventing any ads getting through to my eyeballs. I believe it also helps a bit with security, as it blocks those popups that people click on that install malware...
CommentBlocker
A recent addition. The Internet has often been described as a morass of lies and misinformation. Personally, I think that's a bit overboard. It's not the whole Internet, it's just the comments on most sites. The comments sections of science sites are overrun by anti-science right-wingers hell bent on setting fire to the planet. Those on IT sites are full of ignoramuses who delight in their ignorance of "off" and "on". I always TRIED to not get pulled in to the comment madness that exists at the bottom of every article. But like looking through some kind of Lovecraftian portal, just glimpsing them sucks any sanity right out of my head.
CommentBlocker is your defense against the babbling of the Old One's minions. It literally eliminates the comments section from practically every site I've been on since I started using it. The Internet is a quieter, saner place because of this fantastic extension. If there's a site where you want to see comments, though, it allows you to whitelist them (you can also temporarily whitelist a site while you're looking at it by hitting a button in the address bar). Interestingly, if you run a Wordpress site, it'll actually block the comments administration pages on the back end, something that left me scratching my head for a few minutes when I first encountered it.
FF Rocker
A lifetime ago, I was a fierce Internet Explorer user. In fact, it wasn't until deep into the Firefox 2.x life cycle that I finally switched away (because of the extensions). I didn't use IE by default, though. I used a "skin" called AvantBrowser that added many useful features to my then-beloved browser. Most notably was tabs. Opera Browser was the first to fully implement them, but I found their initial offering clunky and annoying. Avant's implementation seemed to work a lot smoother and wasn't a huge deviation from my current browsing habits.
But, one of the features Avant had that still influences my choice of browser to this day: rocker gestures. In fact, I would say that if I couldn't add this feature to a browser, it would be a showstopper above all others. Once you use it, and get used to it, you just can't live without it. This feature allow you to do away with the Forward and Back buttons. Normally, if you want to go back, you have to move the mouse up to the top of the screen and hit the back button. A minor gesture, yes, but they add up. With rocker gestures, if you want to go back, hold down the right-button on your mouse and hit the left one. If you want to move forward, hold down the left button and hit the right. The mouse is the primary browsing interface and being able to navigate without extra effort just makes things easier.
Trust me, I know how silly it sounds, but if you've got FF, install FF Rocker and try it. After a month, see if you can use a browser without it. :)
The "Shoulds"
Those are the big three. Without them, there's no point in my moving to a new version or browser. They're so ingrained into my browsing habits that without them the web is an annoying, slow to navigate, flashy place with no purpose. This next set represents those that I COULD live without, but would prefer not to. They make browsing more pleasant and safer.
Greasemonkey
The techie's favorite extension. Greasemonkey exists on the premise that regardless of how a web jockey wants to present information to you, how it's ultimately viewed should be up to you. Using small scripts, a trove of which can be found at Userscripts, Greasemonkey will modify how a particular page will act or display in your browser. For example, a lot of people hate the new layout used by Gawker sites like Lifehacker and io9. There exist Greasemonkey scripts that will modify how those sites look to conform more to what is the norm on the web. Nothing changes at Gawker's end, they still think you're seeing their site as they intended. Everything happens at your end.
Beyond visual tweaks, Greasemonkey scripts can add new features or make some sites more usable. For example, there are scripts that can automate the playing of games on Facebook or allow you to download videos from Youtube. One of my faves blocks Flash animations on sites from auto-starting until I tell them to. Another converts any URLs found on a page that haven't been made links by the site's author into clickable links. No cutting and pasting these anomalies. If you like everything "just so", Greasemonkey might be one for you.
IE Tab 2
My workplace pretty much supports IE only. Our internal intranet is fairly biased toward that browser, and no amount of fiddling will get some of them to work at all in Firefox. I could launch IE every time I need to go into one, but I usually have a browser open, so why bother? IE Tab embeds an IE instance in a Firefox tab when you need it. You can setup rules so that any time you go to a particular site or domain, it'll automatically load that page in an IE tab rather than a Firefox one. It's a great little hack to work around those who don't grok the web is supposed to be cross-platform.
Note: there are a number of variations of IE Tab in the extension database. I've found IE Tab 2 to be the only one that consistently works well. Others I've found will lock up the browser on certain sites, others just won't load at all.
HTTPS-Everywhere
A recent addition, and one that will probably move up to the "Musts" category eventually, this extension is from the Electronic Freedom Foundation. There are a number of sites, such as Google, Facebook, Twitter, Yahoo, etc, that contain a large portion of our personal information, yet all of that information is sent back and forth from site to browser over the Internet in clear text. Anyone can intercept your communications and snoop on what you're doing. Some sites have started forcing encrypted communications in recent years (like Gmail), but others have not. A recent exploit, Firesheep, would allow a hacker to sit in his local Starbucks and capture the logins of the other patrons quickly and easily.
Ironically, most of these sites already have the ability to encrypt all communications, but it's not turned on by default. What HTTPS-Everywhere does is force the browser to only use encrypted communications with the sites it knows supports it. The hope is that all sites will someday encrypt every bit of communication. Until that time, this extension will protect you on quite a number of sites.
LastPass
Another recent addtion. I have to confess, that even as an IT pro I'm terrible when it comes to password management. I have a suite of 5 or 6 different passwords I use for different things, and then end up using them everywhere. I've tried this tool and that software, but nothing was ubiquitous, cross-platform and seamless enough for me.
LastPass is a univeral password manager. It integrates into your browser, and on Firefox even replaces the built-in password management system. There are apps for every portable and desktop platform, or you can simply go to the website if you need to get a password when away from your own equipment.
After reading so many articles about LastPass on Lifehacker, I finally decided to weigh the odds...after all, in order to use LastPass, you have to hand over all of your usernames and passwords to someone else to secure for you. That's a huge risk! But, I weighed that against the infinitely less secure method of failing to maintain good password management myself and decided it's the lesser of two evils.
So far, I've been very happy with the service and how it works. It even has a password generator, so when I sign up for a new service I generate a completely new password for it. LastPass stores it and I never know what it is. Easy-peasy!
The next category I'm going to cover are some minor additions I like to have, but also the significant tweaks I make to the appearance and configuration of Firefox itself to meet my needs. But, I'm sure you're as tired reading this as I am writing it. :) Hopefully I'll be better about the follow-up than I have been with others!
On first launch, performance seemed to be noticeably faster than 3.6.13, the previous version I was using. Opening Gmail was near-instantaneous and other sites just popped into view as soon as I finished typing the URL. It only took a few minutes before I decided this was worth checking out further, so on to the next most important step: are my extensions available?
One of the main reasons I use Firefox is the customization through extensions. If the extension author hasn't updated it, that usually restricts if I'm going to move to a new version of the browser or not. This isn't true of all of them, but I do have a suite of "core" extensions I require at all times:
The "Musts"
Adblock Plus
I'm an American. I want everything, and I want it for free, dammit! :) I get it, sites use ads to generate revenue. Some generate a lot of revenue from them, but those sites typically have so many ads on them their content is almost unusable. Adblock Plus does a great job of preventing any ads getting through to my eyeballs. I believe it also helps a bit with security, as it blocks those popups that people click on that install malware...
CommentBlocker
A recent addition. The Internet has often been described as a morass of lies and misinformation. Personally, I think that's a bit overboard. It's not the whole Internet, it's just the comments on most sites. The comments sections of science sites are overrun by anti-science right-wingers hell bent on setting fire to the planet. Those on IT sites are full of ignoramuses who delight in their ignorance of "off" and "on". I always TRIED to not get pulled in to the comment madness that exists at the bottom of every article. But like looking through some kind of Lovecraftian portal, just glimpsing them sucks any sanity right out of my head.
CommentBlocker is your defense against the babbling of the Old One's minions. It literally eliminates the comments section from practically every site I've been on since I started using it. The Internet is a quieter, saner place because of this fantastic extension. If there's a site where you want to see comments, though, it allows you to whitelist them (you can also temporarily whitelist a site while you're looking at it by hitting a button in the address bar). Interestingly, if you run a Wordpress site, it'll actually block the comments administration pages on the back end, something that left me scratching my head for a few minutes when I first encountered it.
FF Rocker
A lifetime ago, I was a fierce Internet Explorer user. In fact, it wasn't until deep into the Firefox 2.x life cycle that I finally switched away (because of the extensions). I didn't use IE by default, though. I used a "skin" called AvantBrowser that added many useful features to my then-beloved browser. Most notably was tabs. Opera Browser was the first to fully implement them, but I found their initial offering clunky and annoying. Avant's implementation seemed to work a lot smoother and wasn't a huge deviation from my current browsing habits.
But, one of the features Avant had that still influences my choice of browser to this day: rocker gestures. In fact, I would say that if I couldn't add this feature to a browser, it would be a showstopper above all others. Once you use it, and get used to it, you just can't live without it. This feature allow you to do away with the Forward and Back buttons. Normally, if you want to go back, you have to move the mouse up to the top of the screen and hit the back button. A minor gesture, yes, but they add up. With rocker gestures, if you want to go back, hold down the right-button on your mouse and hit the left one. If you want to move forward, hold down the left button and hit the right. The mouse is the primary browsing interface and being able to navigate without extra effort just makes things easier.
Trust me, I know how silly it sounds, but if you've got FF, install FF Rocker and try it. After a month, see if you can use a browser without it. :)
The "Shoulds"
Those are the big three. Without them, there's no point in my moving to a new version or browser. They're so ingrained into my browsing habits that without them the web is an annoying, slow to navigate, flashy place with no purpose. This next set represents those that I COULD live without, but would prefer not to. They make browsing more pleasant and safer.
Greasemonkey
The techie's favorite extension. Greasemonkey exists on the premise that regardless of how a web jockey wants to present information to you, how it's ultimately viewed should be up to you. Using small scripts, a trove of which can be found at Userscripts, Greasemonkey will modify how a particular page will act or display in your browser. For example, a lot of people hate the new layout used by Gawker sites like Lifehacker and io9. There exist Greasemonkey scripts that will modify how those sites look to conform more to what is the norm on the web. Nothing changes at Gawker's end, they still think you're seeing their site as they intended. Everything happens at your end.
Beyond visual tweaks, Greasemonkey scripts can add new features or make some sites more usable. For example, there are scripts that can automate the playing of games on Facebook or allow you to download videos from Youtube. One of my faves blocks Flash animations on sites from auto-starting until I tell them to. Another converts any URLs found on a page that haven't been made links by the site's author into clickable links. No cutting and pasting these anomalies. If you like everything "just so", Greasemonkey might be one for you.
IE Tab 2
My workplace pretty much supports IE only. Our internal intranet is fairly biased toward that browser, and no amount of fiddling will get some of them to work at all in Firefox. I could launch IE every time I need to go into one, but I usually have a browser open, so why bother? IE Tab embeds an IE instance in a Firefox tab when you need it. You can setup rules so that any time you go to a particular site or domain, it'll automatically load that page in an IE tab rather than a Firefox one. It's a great little hack to work around those who don't grok the web is supposed to be cross-platform.
Note: there are a number of variations of IE Tab in the extension database. I've found IE Tab 2 to be the only one that consistently works well. Others I've found will lock up the browser on certain sites, others just won't load at all.
HTTPS-Everywhere
A recent addition, and one that will probably move up to the "Musts" category eventually, this extension is from the Electronic Freedom Foundation. There are a number of sites, such as Google, Facebook, Twitter, Yahoo, etc, that contain a large portion of our personal information, yet all of that information is sent back and forth from site to browser over the Internet in clear text. Anyone can intercept your communications and snoop on what you're doing. Some sites have started forcing encrypted communications in recent years (like Gmail), but others have not. A recent exploit, Firesheep, would allow a hacker to sit in his local Starbucks and capture the logins of the other patrons quickly and easily.
Ironically, most of these sites already have the ability to encrypt all communications, but it's not turned on by default. What HTTPS-Everywhere does is force the browser to only use encrypted communications with the sites it knows supports it. The hope is that all sites will someday encrypt every bit of communication. Until that time, this extension will protect you on quite a number of sites.
LastPass
Another recent addtion. I have to confess, that even as an IT pro I'm terrible when it comes to password management. I have a suite of 5 or 6 different passwords I use for different things, and then end up using them everywhere. I've tried this tool and that software, but nothing was ubiquitous, cross-platform and seamless enough for me.
LastPass is a univeral password manager. It integrates into your browser, and on Firefox even replaces the built-in password management system. There are apps for every portable and desktop platform, or you can simply go to the website if you need to get a password when away from your own equipment.
After reading so many articles about LastPass on Lifehacker, I finally decided to weigh the odds...after all, in order to use LastPass, you have to hand over all of your usernames and passwords to someone else to secure for you. That's a huge risk! But, I weighed that against the infinitely less secure method of failing to maintain good password management myself and decided it's the lesser of two evils.
So far, I've been very happy with the service and how it works. It even has a password generator, so when I sign up for a new service I generate a completely new password for it. LastPass stores it and I never know what it is. Easy-peasy!
The next category I'm going to cover are some minor additions I like to have, but also the significant tweaks I make to the appearance and configuration of Firefox itself to meet my needs. But, I'm sure you're as tired reading this as I am writing it. :) Hopefully I'll be better about the follow-up than I have been with others!
Monday, November 22, 2010
Dropbox: spreadin' the love, gettin' the space
Today a couple of odd posts showed up on my Facebook and Twitter pages about my favorite online service, Dropbox. (Note: the previous link is a referral link to Dropbox. More on this in a bit.) This is because Dropbox recently put up a page where folks could get up to an extra 768M of space by doing some simple social media connection stuff.
I've talked about Dropbox before, but it still never ceases to amaze me how many times I find new uses for it. Let's look at a few of the ones I've been using:
- There's a folder called "Torrents" in my Dropbox. I've also installed Dropbox on the server I have at my house running uTorrent. uTorrent monitors this folder and when new torrents show up there, it starts downloading them. So, I just download a torrent into that folder from any machine I'm logged in to and I'm good to go!
- If I need to get a file into my new EC2-hosted Ubuntu box, no worries. I've got the text-based install on it and I just drop them into there. Also, if I need to do major edits on config files, I can get them to my desktop quickly.
- Thanks to some of Dropbox' partners, I essentially have an extra 4G on my iPhone now. More and more mobile apps are using Dropbox for back end storage because it's easily accessed and ubiquitous. I've held off until now, but I'm finally upgrading to iOS 4 simply to give Picbox a try. But for now...
- When I have a quiet moment at the office, I plug my iPhone into my laptop and dump all current photos from the phone into my Dropbox folder so they're ready to be organized when I get home.
- Still in the works: I've got Portable Firefox out there. I've put in a master password, so my passwords are encrypted and thus protected from prying eyes @ Dropbox. I'm in the process of setting up the instance of Firefox on my laptop to use the profile directory that's in the Dropbox folder so I have one universal Firefox profile across machines.
- When I first started my new job, I had some coding work that they needed me to do for my old team. Unfortunately, I didn't get Visual Studio with my new machine and didn't have rights to install it. So, I used an online service to take control of my home machine's desktop and did my changes there. Once the code was compiled, I plopped the executable into my Dropbox and was able to get it back to my team.
- My wife doesn't have a lot of data she keeps on her machine. In fact, almost all of it fits within her 2G of space, so I have her keep all of her files in the Dropbox folder on her machine. Tada! All of her files are backed up as soon as they're created!
- Dropbox also keeps old versions of files. There's been a couple of times where I saved a file and overwrote an original I didn't want to change. No biggie, find the file on the website and click "Previous versions" to get the older instance back. Lifesaver!
I'm sure you can find plenty of other uses yourself or at least find dozens of solutions other folks have come up with. I've looked at some of the other services that are out there, but I just keep coming back to Dropbox for its ease of use and ubiquity. Here's a quick start guide to get you up and running with tons of space as quickly as possible:
1. Hit Dropbox' site. That link is a referral which will give both you and I an extra 250M of space in our accounts once you've installed the software.
2. Download and install a client on your machine. At this point, we each get our extra space. Total space: 2.25G
3. Revisit your Dropbox home page. At the top of the page, you'll see a set of tabs (Getting Started, Files, Events, Sharing, Help). That Getting Started tab is where we're headed next. Go through the steps listed and you'll get an extra 250M of space! You don't have to do all of them, I'd start with 1, 2, 3, & 5. I think those were all I needed to do to get the extra 250M. Total space: 2.5G.
4. Head to the new Free page and do each of the steps listed. Once you're done, you'll get up to an extra 768M of space for free. If you're not happy about exposing everything to Dropbox on Facebook or Twitter, simply deactivate the application's access once you're done. The free space will stay. Total space: 3.328G.
5. Get referrals. At that link, you'll find your referral link that you can give out to friends and family for more space. You can get up to an extra 10G of space this way. Please be aware of some etiquette. There are a number of sites like Lifehacker that will ban you if you post your referral link in their comments. A lot of people take advantage of their Dropbox-based posts to try and drum up some space, and the comments become useless because of it.
Also, the URLs posted to your Facebook and Twitter pages after you do step 4? All of them are referral links to your account. So, once you've done that step, you've also put out a couple of requests for referrals. How cool is that?
I've talked about Dropbox before, but it still never ceases to amaze me how many times I find new uses for it. Let's look at a few of the ones I've been using:
- There's a folder called "Torrents" in my Dropbox. I've also installed Dropbox on the server I have at my house running uTorrent. uTorrent monitors this folder and when new torrents show up there, it starts downloading them. So, I just download a torrent into that folder from any machine I'm logged in to and I'm good to go!
- If I need to get a file into my new EC2-hosted Ubuntu box, no worries. I've got the text-based install on it and I just drop them into there. Also, if I need to do major edits on config files, I can get them to my desktop quickly.
- Thanks to some of Dropbox' partners, I essentially have an extra 4G on my iPhone now. More and more mobile apps are using Dropbox for back end storage because it's easily accessed and ubiquitous. I've held off until now, but I'm finally upgrading to iOS 4 simply to give Picbox a try. But for now...
- When I have a quiet moment at the office, I plug my iPhone into my laptop and dump all current photos from the phone into my Dropbox folder so they're ready to be organized when I get home.
- Still in the works: I've got Portable Firefox out there. I've put in a master password, so my passwords are encrypted and thus protected from prying eyes @ Dropbox. I'm in the process of setting up the instance of Firefox on my laptop to use the profile directory that's in the Dropbox folder so I have one universal Firefox profile across machines.
- When I first started my new job, I had some coding work that they needed me to do for my old team. Unfortunately, I didn't get Visual Studio with my new machine and didn't have rights to install it. So, I used an online service to take control of my home machine's desktop and did my changes there. Once the code was compiled, I plopped the executable into my Dropbox and was able to get it back to my team.
- My wife doesn't have a lot of data she keeps on her machine. In fact, almost all of it fits within her 2G of space, so I have her keep all of her files in the Dropbox folder on her machine. Tada! All of her files are backed up as soon as they're created!
- Dropbox also keeps old versions of files. There's been a couple of times where I saved a file and overwrote an original I didn't want to change. No biggie, find the file on the website and click "Previous versions" to get the older instance back. Lifesaver!
I'm sure you can find plenty of other uses yourself or at least find dozens of solutions other folks have come up with. I've looked at some of the other services that are out there, but I just keep coming back to Dropbox for its ease of use and ubiquity. Here's a quick start guide to get you up and running with tons of space as quickly as possible:
1. Hit Dropbox' site. That link is a referral which will give both you and I an extra 250M of space in our accounts once you've installed the software.
2. Download and install a client on your machine. At this point, we each get our extra space. Total space: 2.25G
3. Revisit your Dropbox home page. At the top of the page, you'll see a set of tabs (Getting Started, Files, Events, Sharing, Help). That Getting Started tab is where we're headed next. Go through the steps listed and you'll get an extra 250M of space! You don't have to do all of them, I'd start with 1, 2, 3, & 5. I think those were all I needed to do to get the extra 250M. Total space: 2.5G.
4. Head to the new Free page and do each of the steps listed. Once you're done, you'll get up to an extra 768M of space for free. If you're not happy about exposing everything to Dropbox on Facebook or Twitter, simply deactivate the application's access once you're done. The free space will stay. Total space: 3.328G.
5. Get referrals. At that link, you'll find your referral link that you can give out to friends and family for more space. You can get up to an extra 10G of space this way. Please be aware of some etiquette. There are a number of sites like Lifehacker that will ban you if you post your referral link in their comments. A lot of people take advantage of their Dropbox-based posts to try and drum up some space, and the comments become useless because of it.
Also, the URLs posted to your Facebook and Twitter pages after you do step 4? All of them are referral links to your account. So, once you've done that step, you've also put out a couple of requests for referrals. How cool is that?
Thursday, November 18, 2010
The new "paper or plastic"
A friend recently posted the question on his Facebook page "Nook or Kindle?". Some folks offered up useful advice, but just as many felt the need to chime in with "REAL BOOKS!!" Sigh...
I often wonder about people who get upset when new technologies supplant the old. Do they still have rotary phones? Hell, do they still have a land line? When Verizon stops automatic delivery of phone books, will they call to get one? It's been a decade since I've even had one of those in my house. When I come home and find them on the porch, they go right into the recycling bin. I have an iPhone, why do I need a phone book? To quote the song, I've got the whole world in my hand. These strike me as the same kind of people who would've dwelt on the "loss" to society when cars replaced horse-drawn carriages.
As someone who is a habitual pack rat, even I see the value in not holding on to the past. My comic books are stored in digital format, along with my documents, ebooks, pictures and movies. As pointed out in an earlier article, if there's a fire in my house, I won't lose a single picture of my daughter. Can the Luddites say the same?
Look, I get it. There's a certain romanticism about books. Back when I had time in my days, I used to read as well. I love to read, but the reality is, actual books are inconvenient. Electronic books and publishing is the future, so you might as well start dealing with it. The last couple of books I've purchased were both ebooks. I read them on my iPhone when I had a moment or two. There are times I feel like putting the effort into finding pirated digital copies of every book I have and ditching the paperware versions. It would certainly make my wife happy, and certainly make moving a whole lot easier when the time comes.
And since some will wonder, my response to my friend's question was: iPad. Before the "fanboi" accusations start flying in, yes, I have an iPhone. But, I won't be going back to an iOS device for my next phone, I'm going Android. The Apple platform's just gotten far too restrictive and I have no interest in dealing with it anymore. However, an iPad wouldn't be my primary device so I could live with some level of restriction there. For my friend, for just a little more than the cost of either dedicated device, he could get an iPad, install both the free Nook and Kindle apps as well as a host of other apps and get a much better experience. I've seen the iPad's book readers, they're fantastic. The iBook app is brilliant, you really feel like you're turning a page with it. Personally, I'm waiting for the second gen of the iPad (With a camera, please, Steve! Seriously, what were you thinking?), but there's no reason the first gen can' t be a primary ereader.
Nope, I know I'm a technophile and I know others are not. But, when someone's making a decision between two new technologies, they're obviously not interested in hearing that they're destroying civilization and should stick with the old stuff. Because, really, that's what these people were saying. These kids these days and their new-fangled ebooks...
I often wonder about people who get upset when new technologies supplant the old. Do they still have rotary phones? Hell, do they still have a land line? When Verizon stops automatic delivery of phone books, will they call to get one? It's been a decade since I've even had one of those in my house. When I come home and find them on the porch, they go right into the recycling bin. I have an iPhone, why do I need a phone book? To quote the song, I've got the whole world in my hand. These strike me as the same kind of people who would've dwelt on the "loss" to society when cars replaced horse-drawn carriages.
As someone who is a habitual pack rat, even I see the value in not holding on to the past. My comic books are stored in digital format, along with my documents, ebooks, pictures and movies. As pointed out in an earlier article, if there's a fire in my house, I won't lose a single picture of my daughter. Can the Luddites say the same?
Look, I get it. There's a certain romanticism about books. Back when I had time in my days, I used to read as well. I love to read, but the reality is, actual books are inconvenient. Electronic books and publishing is the future, so you might as well start dealing with it. The last couple of books I've purchased were both ebooks. I read them on my iPhone when I had a moment or two. There are times I feel like putting the effort into finding pirated digital copies of every book I have and ditching the paperware versions. It would certainly make my wife happy, and certainly make moving a whole lot easier when the time comes.
And since some will wonder, my response to my friend's question was: iPad. Before the "fanboi" accusations start flying in, yes, I have an iPhone. But, I won't be going back to an iOS device for my next phone, I'm going Android. The Apple platform's just gotten far too restrictive and I have no interest in dealing with it anymore. However, an iPad wouldn't be my primary device so I could live with some level of restriction there. For my friend, for just a little more than the cost of either dedicated device, he could get an iPad, install both the free Nook and Kindle apps as well as a host of other apps and get a much better experience. I've seen the iPad's book readers, they're fantastic. The iBook app is brilliant, you really feel like you're turning a page with it. Personally, I'm waiting for the second gen of the iPad (With a camera, please, Steve! Seriously, what were you thinking?), but there's no reason the first gen can' t be a primary ereader.
Nope, I know I'm a technophile and I know others are not. But, when someone's making a decision between two new technologies, they're obviously not interested in hearing that they're destroying civilization and should stick with the old stuff. Because, really, that's what these people were saying. These kids these days and their new-fangled ebooks...
Tuesday, November 16, 2010
Top o' the EC2 ya!
So, things have been a little quiet here lately. Most of the blame can be placed on TEDxRochester, of course. But, now that it's over, life is slowly starting to return to normal and I'm returning back to my projects. As you saw from my last post, I still continue playing around with Ubuntu, but aside from just my laptop getting my attention I've added a new Ubuntu project.
A few weeks ago, I learned that Amazon was opening up a free tier to their Amazon Web Services. Now, I haven't wrapped my whole head around the platform yet, but it really is an amazing suite of products and offerings the big "A" has put out there. In a nutshell, AWS is a virtual hosting service, commonly referred to as a cloud service.
AWS is made up of a couple of offerings, but the one I'm currently spending the most time on is the Amazon Elastic Compute Cloud (Amazon EC2), their virtual machine hosting service. I've actually got a virtual machine running Ubuntu Server out on their hardware. In fact, you're visiting it right now! Amazon, and other members of the AWS community, provide base images that can then be built on to establish your own presence on the web. As an example, this machine is built from AMI-00877069. This is a Canonical-provided Amazon machine image (AMI) of the Maverick Meerkat (10.10) version of Ubuntu Server. Simply select your AMI, launch your instance and you've got a machine in the cloud!
From there I added Apache, MySQL, Wordpress, etc. These machines are not accessible from the Internet immediately, though. They exist on a private 10.x.x.x subnet, but Amazon provides what they call their "Flexible IPs" which are nothing more than externally accessible IPs that you can then map to your "box". What's even nicer is Amazon provides you with a dead-simple firewall setup to decide how accessible your systems are. You can create groups to segment ("Web" group is accessible on port 80, but incoming mail servers on 25 only) and open up only what you need.
Since I'd already registered the domain, all I needed to do was modify the DNS at my registrar and I was on the web! In fact, if you want to visit my wife's site and get yourself a zombie portrait done, that too is hosted on this virtual box.
As mentioned, I haven't really dug too deeply into the full offering set, but as I find need I check it out. Most of the services are geared to the web developer/web applications crowd. For example, the SimpleDB service allows you to host your database at Amazon without having to worry about the details of administering a database server. Put your data in, pull it out as needed. Next on my list to work with is S3, their cloud storage solution. There's apparently a bug in their console that makes it inaccessible to browsers using the newest Flash, and so I haven't fiddled much with it.
So, aside from giving me yet another toy to play with, what's the market for Amazon Web Services? The answer: the sky's the limit. I first became aware of EC2, for example, when checking out web hosts for the domains my wife and I needed. A number of the more inexpensive hosts proudly boast that they're hosted at Amazon. Quite a few of the newer "Web 2.0" (are we still using that term?) services also are hosted here. Think about it: you want to run a web host or a web-based application, but you don't want the hassle of a server farm. With Amazon Web Services, you simply fire off instances as needed and manage your entire infrastructure from your living room in your PJs.
You also can provide your customers with a level of scalability that's difficult for even the most experienced admins to provide (said as an admin who's had to provide those levels of scalability!) You can, for example, start with a single instance to handle traffic. As traffic expands, you can configure AWS to fire off new instances of your hosts to handle the load. As the load decreases, the instances are shut down. How cool is that?
I've spent some time in my career working with VMware and Citrix XenServer farms, and Amazon Web Services is that to the next level. In the IT world, virtualization's the big buzz now as you can provide your end users with scalability and availability that was never before possible. Amazon's made it easy for anyone to tap into that power with nothing more than a web browser. But, beyond all of that, the services are CHEAP! A "small" instance, which is a VM with 1.7G of memory will run you about 8.5 cents/hour. That's about $60/month. You can barely lease a physical server for that much, and when you add in costs for electricity and maintenance, Amazon Web Services is a steal.
As I mentioned, I'm taking advantage of the free tier that's being offered right now. New AWS customers can get on this tier for 12 months. This includes:
- 750 hours of Amazon EC2 Linux Micro Instance usage (613 MB of memory and 32-bit and 64-bit platform support) – enough hours to run continuously each month
- 750 hours of an Elastic Load Balancer plus 15 GB data processing
- 10 GB of Amazon Elastic Block Storage, plus 1 million I/Os, 1 GB of snapshot storage, 10,000 snapshot Get Requests and 1,000 snapshot Put Requests
- 5 GB of Amazon S3 storage, 20,000 Get Requests, and 2,000 Put Requests
- 30 GB per of internet data transfer (15 GB of data transfer “in” and 15 GB of data transfer “out” across all services except Amazon CloudFront)
- 25 Amazon SimpleDB Machine Hours and 1 GB of Storage
- 100,000 Requests of Amazon Simple Queue Service
- 100,000 Requests, 100,000 HTTP notifications and 1,000 email notifications for Amazon Simple Notification Service
Now, even if you're a seasoned IT pro you're probably wondering what some of those things mean. The reality is, Amazon has come up with their own phrasing and terms for some things, so you really have to dig in yourself to see. For me, though, having that free virtual machine is more than enough reason to give it a try! Keep in mind, though, it's free for a year. After that, it's 2 cents/hour to run your box. Oh, my, 2 cents/hour! That's $15/month! Told you...holy crap this is a cheap service!
Now, if you're going to sign up, learn from one of my big mistakes: don't just grab an AMI to launch. There are two types of instances you can use, a standard EC2 one and one that's EBS-backed (Elastic Block Storage, part of the Amazon S3 services). Non-EBS instances are temporary only. If you shut the machine down, even to reboot, it goes away permanently. It's designed for scenarios like I mentioned above: firing up new, temporary instances for handling load. Is there anything more fun than rebooting to install updates only to have all of the work you've done vanish into the ether? Yeah, pay attention to your AMI types, people!
When you choose to launch your first instance, select the "Community AMIs" tab and select "EBS Images" in the dropdown and go from there. If you're going to try out Ubuntu, stick with the official Canonical-provided images. These all come from user 099720109477, so you can just search for them. You need to remember to add an Elastic IP to it and at least open up port 22 so you can SSH in later, but beyond that...you're done (assuming you remember to make it a Tiny instance). Build your box and go. Don't want to be limited to Server? No reason you can't build a full desktop accessible everywhere. :)
You also have options to run Windows boxes, but you cannot get them in a Tiny instance, so they're not free. But, it's doable! As I learn more about the services and offerings, I'll be posting my findings and learnings. I've also learned tons about managing Apache, MySQL and Wordpress that I'm looking to share, too. For now, give it a try! It's FREE!!
A few weeks ago, I learned that Amazon was opening up a free tier to their Amazon Web Services. Now, I haven't wrapped my whole head around the platform yet, but it really is an amazing suite of products and offerings the big "A" has put out there. In a nutshell, AWS is a virtual hosting service, commonly referred to as a cloud service.
AWS is made up of a couple of offerings, but the one I'm currently spending the most time on is the Amazon Elastic Compute Cloud (Amazon EC2), their virtual machine hosting service. I've actually got a virtual machine running Ubuntu Server out on their hardware. In fact, you're visiting it right now! Amazon, and other members of the AWS community, provide base images that can then be built on to establish your own presence on the web. As an example, this machine is built from AMI-00877069. This is a Canonical-provided Amazon machine image (AMI) of the Maverick Meerkat (10.10) version of Ubuntu Server. Simply select your AMI, launch your instance and you've got a machine in the cloud!
From there I added Apache, MySQL, Wordpress, etc. These machines are not accessible from the Internet immediately, though. They exist on a private 10.x.x.x subnet, but Amazon provides what they call their "Flexible IPs" which are nothing more than externally accessible IPs that you can then map to your "box". What's even nicer is Amazon provides you with a dead-simple firewall setup to decide how accessible your systems are. You can create groups to segment ("Web" group is accessible on port 80, but incoming mail servers on 25 only) and open up only what you need.
Since I'd already registered the domain, all I needed to do was modify the DNS at my registrar and I was on the web! In fact, if you want to visit my wife's site and get yourself a zombie portrait done, that too is hosted on this virtual box.
As mentioned, I haven't really dug too deeply into the full offering set, but as I find need I check it out. Most of the services are geared to the web developer/web applications crowd. For example, the SimpleDB service allows you to host your database at Amazon without having to worry about the details of administering a database server. Put your data in, pull it out as needed. Next on my list to work with is S3, their cloud storage solution. There's apparently a bug in their console that makes it inaccessible to browsers using the newest Flash, and so I haven't fiddled much with it.
So, aside from giving me yet another toy to play with, what's the market for Amazon Web Services? The answer: the sky's the limit. I first became aware of EC2, for example, when checking out web hosts for the domains my wife and I needed. A number of the more inexpensive hosts proudly boast that they're hosted at Amazon. Quite a few of the newer "Web 2.0" (are we still using that term?) services also are hosted here. Think about it: you want to run a web host or a web-based application, but you don't want the hassle of a server farm. With Amazon Web Services, you simply fire off instances as needed and manage your entire infrastructure from your living room in your PJs.
You also can provide your customers with a level of scalability that's difficult for even the most experienced admins to provide (said as an admin who's had to provide those levels of scalability!) You can, for example, start with a single instance to handle traffic. As traffic expands, you can configure AWS to fire off new instances of your hosts to handle the load. As the load decreases, the instances are shut down. How cool is that?
I've spent some time in my career working with VMware and Citrix XenServer farms, and Amazon Web Services is that to the next level. In the IT world, virtualization's the big buzz now as you can provide your end users with scalability and availability that was never before possible. Amazon's made it easy for anyone to tap into that power with nothing more than a web browser. But, beyond all of that, the services are CHEAP! A "small" instance, which is a VM with 1.7G of memory will run you about 8.5 cents/hour. That's about $60/month. You can barely lease a physical server for that much, and when you add in costs for electricity and maintenance, Amazon Web Services is a steal.
As I mentioned, I'm taking advantage of the free tier that's being offered right now. New AWS customers can get on this tier for 12 months. This includes:
- 750 hours of Amazon EC2 Linux Micro Instance usage (613 MB of memory and 32-bit and 64-bit platform support) – enough hours to run continuously each month
- 750 hours of an Elastic Load Balancer plus 15 GB data processing
- 10 GB of Amazon Elastic Block Storage, plus 1 million I/Os, 1 GB of snapshot storage, 10,000 snapshot Get Requests and 1,000 snapshot Put Requests
- 5 GB of Amazon S3 storage, 20,000 Get Requests, and 2,000 Put Requests
- 30 GB per of internet data transfer (15 GB of data transfer “in” and 15 GB of data transfer “out” across all services except Amazon CloudFront)
- 25 Amazon SimpleDB Machine Hours and 1 GB of Storage
- 100,000 Requests of Amazon Simple Queue Service
- 100,000 Requests, 100,000 HTTP notifications and 1,000 email notifications for Amazon Simple Notification Service
Now, even if you're a seasoned IT pro you're probably wondering what some of those things mean. The reality is, Amazon has come up with their own phrasing and terms for some things, so you really have to dig in yourself to see. For me, though, having that free virtual machine is more than enough reason to give it a try! Keep in mind, though, it's free for a year. After that, it's 2 cents/hour to run your box. Oh, my, 2 cents/hour! That's $15/month! Told you...holy crap this is a cheap service!
Now, if you're going to sign up, learn from one of my big mistakes: don't just grab an AMI to launch. There are two types of instances you can use, a standard EC2 one and one that's EBS-backed (Elastic Block Storage, part of the Amazon S3 services). Non-EBS instances are temporary only. If you shut the machine down, even to reboot, it goes away permanently. It's designed for scenarios like I mentioned above: firing up new, temporary instances for handling load. Is there anything more fun than rebooting to install updates only to have all of the work you've done vanish into the ether? Yeah, pay attention to your AMI types, people!
When you choose to launch your first instance, select the "Community AMIs" tab and select "EBS Images" in the dropdown and go from there. If you're going to try out Ubuntu, stick with the official Canonical-provided images. These all come from user 099720109477, so you can just search for them. You need to remember to add an Elastic IP to it and at least open up port 22 so you can SSH in later, but beyond that...you're done (assuming you remember to make it a Tiny instance). Build your box and go. Don't want to be limited to Server? No reason you can't build a full desktop accessible everywhere. :)
You also have options to run Windows boxes, but you cannot get them in a Tiny instance, so they're not free. But, it's doable! As I learn more about the services and offerings, I'll be posting my findings and learnings. I've also learned tons about managing Apache, MySQL and Wordpress that I'm looking to share, too. For now, give it a try! It's FREE!!
Sunday, November 14, 2010
Lazyman's back up - the only kind that works! (Part 1)
Backup and backup often, right? We all do it, don't we? Yeah, I didn't do it for a while, either. I did get better at it once my daughter was born. Those pictures of her first few years of her life are irreplaceable, so I became diligent in ensuring they were as safe as I could make them. Since I had a machine in our living room that served as our media center and file server, I would backup to that. When out of town, I'd always have my laptop with me to serve as my "offsite backup".
The problem was, I did everything manually and that's the fastest way into Mistakeville I've found (Most folks take I-380S and then get off at Exit 19. That's the long way). Whenever I'd want to backup, I'd look to see what backups I had out there on the network. Since I did it haphazardly, I'd have a few different copies of previous dumps from my laptop. Since I never knew what was the most current version, I'd just create a new dump. At one point I had about a dozen copies of the same data occupying half a terabyte of drive space.
It was absurd, and I one day I finally decided to get it under control. I went through and consolidated them as best I could as I MOVED them from the server onto my laptop. At that point, the laptop contained the master copy of my data. I then used Microsoft's Synctoy to sync it back to the server. Synctoy gets a bad rap from folks, but it really is a simple and easy to use tool. I created a sync pair for my data and once a week scheduled it to run automatically. I kept my backups down to just one copy and life was good.
Fast forward to today. New OS means new paradigms. Since the laptop's been traveling with me a bit more, using it as an off site backup still works. But, there might come a time where I'll want to leave it at home when I'm at work. Worse, if there's a disaster in the house (more on this topic at another time), I'm probably not going to make an effort to get my laptop! So, I need an additional backup I can keep permanently out of the house. Since I had an extra 200G hard drive lying around, and that's more than enough to hold my REALLY critical data, I ordered an inexpensive USB enclosure to put it in. Once a week, I'll bring it into the house and do a backup, but the remainder of the time it will live in the car. I need to figure out how to handle the very cold winters we have 'round here, though. A week in the car could get it down to absolute zero!
That's a detail for another day, though. Today, we just want to get it setup so I can use it to backup my laptop quickly, easily AND securely. This drive's going to be in my car, so there's always a chance it could "disappear" one day. I don't want my data so easily absconded with! Fortunately, there's an easy solution: Truecrypt. A free, open source encryption application. It's cross-platform, so no matter where my brain leads me in terms of OSes in the future, I'll be able to read it.
Installing on Ubuntu's a doddle: download from here, follow the instructions. Now we create an encrypted volume. I'm not going to take you through this step as Truecrypt's docs are top-notch. I will mention that I created the volume on my Windows box and I did so for a reason: I wanted it formatted as NTFS. My plan is for this to be accessible regardless of the platform or machine I plug it into. NTFS support has become much more ubiquitous, so that's the route I went. I could've probably gotten it formatted with Ubuntu, but it was simplest to just plug it into a Windows box and do it. The final detail: create a keyfile for the disk and tell Truecrypt to use it. You'll see why later, but for now Truecrypt's docs on keyfiles are the best place for a howto.
Everything's in place, now we just need to make it really easy to use. First, I create a couple of scripts to automate the process with. I'll be writing a number of scripts to manage my machine with, so I'm creating a .bin directory in my home directory. Since a lot of these scripts will be for automated processes, I'm going to hold off adding it to my PATH for the moment. Also, I'm putting it in my home directory instead of /usr/bin for a reason which you'll see with the first one I write for this process.
We start it with: joe ~/.bin/mountrose
Yes, I use joe. I like joe. You also now have one question and one complaint, I think: the maker of my external drive enclosure is named Rosewill and I'm not worried about putting the password for the volume in the script because my home directory is encrypted. If someone were to steal my laptop and the enclosure, they'd have to decrypt my home directory to get this script to get my password to get it open. Of course, at that point, they have the primary of the data anyway. The makers of Truecrypt do make a good point we have to acknowledge. If someone should get onto my machine while it's backing up, they could see the whole command line passed to the Truecrypt process and thus be able to get it that way. Again, at that point my machine's been compromised while I'm logged in with the home directory decrypted. I'm done no matter what. Since the plan is for this to be plugged in once a week for only a few minutes, I'm not as concerned about this particular security issue.
So, this script mounts the encrypted partition we created before and then wipes history to make sure there isn't a trail with my password, just in case. When I started playing around with this, I thought I'd be able to just use the password option on the command line, but I found it absolutely would not work unless I put all of the other options in, including using a keyfile. I don't know why that is, nor do I feel like digging far to find out. The remaining two options included:
--protect-hidden=no tells Truecrypt that there's no hidden partition it needs to be concerned about
--fs-options="uid=1000,gid=1000" tells it to set the newly mounted volume as owned by my user account so I can work with the files.
We need another command to unmount the disk when backup's done. As mentioned above, I'm not adding my personal .bin directory to my path until I'm sure I need it, and the command to unmount is only one line, so I decide instead to create a Bash alias. Append the following to the end of your .bash_aliases file:
alias urose="truecrypt -d /mnt/rosewill"
Now we have the mounting and unmounting of the disk automated. Of course, being the lazy fellow I am, I certainly don't want to have to TYPE these commands when I want to back up. I want it to do even that trivial task for me. S'okay. Computers are here to do our work for us, remember? Besides, having the computer do this means I literally have to remember to plug the thing in once a week. Hmmmm...I think I just came up with an excuse to build a robot!
Back to the matter...how do we automate this process? Fortunately, this part's trivial. We write what's known as a udev rule. udev is the Linux subsystem that detects hardware and creates device nodes for them. When you plug in a device hot, udev reads its pre-defined rules to determine what to do with it. We're just adding one of our own that's particular to this single, unique device. First, we find out the perts* about the drive by plugging it in, turning it on and using udevadm to find out more info:
udevadm info -a --name=/dev/sdb
This will give you TONS of useful information about your drive, but we only need a little bit. Specifically, I need to know the manufacturer and model number of what just got plugged in so I ensure it's the disk I want and not some USB flash drive or something. Now, you'll notice this info is broken up into sections that begin "looking at parent device". This is because udevadm looks at the device and then everything that links this device to the OS. Since this is a USB enclosure, that means: the IDE-USB interface, the USB interface, the USB hub, the USB-PCI bridge, etc, etc. I'm most interested in the section below. Keep in mind, when you're writing udev rules, the attributes you're looking at must all reside in one of these blocks. You can't pull the vendor attribute from the SUBSYSTEMS="usb" section and the model from the SUBSYSTEMS="scsi" one. The reason I chose the block below is because that block had all of the relevant data I need.
We now have all we need to create a rule. First, su and joe /etc/udev/rules.d/10-local.rules. We only need one line for this:
What that's saying is "when you see a device that the kernel labels a scsi disk, from the vendor Initio with this model number symlink its devnode as rosewill. Also, since this is a disk, symlink the devnodes for each partition as rosewill1, rosewill2, etc and then run the script /home/akarakas/.bin/mountrose using /bin/sh".
So, why do we do it like this? Let's say, for example, I've plugged in a USB flash drive prior to plugging in this hard drive. There's a chance that flash drive might become /dev/sdb and my backup hard drive /dev/sdc. Can't really write a script for variable disk names, now can I? By setting up the udev rule in this way, this disk will ALWAYS be accessible via /dev/rosewill and its partitions via /dev/rosewillX. I don't need to poke and prod as part of my script, I'm always going to have the right devnode. As to why I launch the script this way, for some reason a bash script doesn't honor the shebang properly when run by udev. I could not find any way to run this script by simply calling it. No biggie, it works.
And now, the moment of truth...plug it in, turn it on and...../mnt/rosewill is mounted, and I can see and modify the files. Woo hoo! Now we have my media setup! Kewl!
At this point, I'm going to conclude this article and leave the remainder for Part 2. In that article, I'll present the full script the system runs when the disk is plugged in and reasons for some of the choices I've made.
*perts = Pertinent details. I'm trying it out, what do you think?
The problem was, I did everything manually and that's the fastest way into Mistakeville I've found (Most folks take I-380S and then get off at Exit 19. That's the long way). Whenever I'd want to backup, I'd look to see what backups I had out there on the network. Since I did it haphazardly, I'd have a few different copies of previous dumps from my laptop. Since I never knew what was the most current version, I'd just create a new dump. At one point I had about a dozen copies of the same data occupying half a terabyte of drive space.
It was absurd, and I one day I finally decided to get it under control. I went through and consolidated them as best I could as I MOVED them from the server onto my laptop. At that point, the laptop contained the master copy of my data. I then used Microsoft's Synctoy to sync it back to the server. Synctoy gets a bad rap from folks, but it really is a simple and easy to use tool. I created a sync pair for my data and once a week scheduled it to run automatically. I kept my backups down to just one copy and life was good.
Fast forward to today. New OS means new paradigms. Since the laptop's been traveling with me a bit more, using it as an off site backup still works. But, there might come a time where I'll want to leave it at home when I'm at work. Worse, if there's a disaster in the house (more on this topic at another time), I'm probably not going to make an effort to get my laptop! So, I need an additional backup I can keep permanently out of the house. Since I had an extra 200G hard drive lying around, and that's more than enough to hold my REALLY critical data, I ordered an inexpensive USB enclosure to put it in. Once a week, I'll bring it into the house and do a backup, but the remainder of the time it will live in the car. I need to figure out how to handle the very cold winters we have 'round here, though. A week in the car could get it down to absolute zero!
That's a detail for another day, though. Today, we just want to get it setup so I can use it to backup my laptop quickly, easily AND securely. This drive's going to be in my car, so there's always a chance it could "disappear" one day. I don't want my data so easily absconded with! Fortunately, there's an easy solution: Truecrypt. A free, open source encryption application. It's cross-platform, so no matter where my brain leads me in terms of OSes in the future, I'll be able to read it.
Installing on Ubuntu's a doddle: download from here, follow the instructions. Now we create an encrypted volume. I'm not going to take you through this step as Truecrypt's docs are top-notch. I will mention that I created the volume on my Windows box and I did so for a reason: I wanted it formatted as NTFS. My plan is for this to be accessible regardless of the platform or machine I plug it into. NTFS support has become much more ubiquitous, so that's the route I went. I could've probably gotten it formatted with Ubuntu, but it was simplest to just plug it into a Windows box and do it. The final detail: create a keyfile for the disk and tell Truecrypt to use it. You'll see why later, but for now Truecrypt's docs on keyfiles are the best place for a howto.
Everything's in place, now we just need to make it really easy to use. First, I create a couple of scripts to automate the process with. I'll be writing a number of scripts to manage my machine with, so I'm creating a .bin directory in my home directory. Since a lot of these scripts will be for automated processes, I'm going to hold off adding it to my PATH for the moment. Also, I'm putting it in my home directory instead of /usr/bin for a reason which you'll see with the first one I write for this process.
We start it with: joe ~/.bin/mountrose
#!/bin/sh
truecrypt /dev/rosewill1 /mnt/rosewill --password=ComplexPassword --keyfiles=/home/tonyk/.bin/rosekey --protect-hidden=no --fs-options="uid=1000,gid=1000"
rm -rf ~/.bash_history
history -c
Yes, I use joe. I like joe. You also now have one question and one complaint, I think: the maker of my external drive enclosure is named Rosewill and I'm not worried about putting the password for the volume in the script because my home directory is encrypted. If someone were to steal my laptop and the enclosure, they'd have to decrypt my home directory to get this script to get my password to get it open. Of course, at that point, they have the primary of the data anyway. The makers of Truecrypt do make a good point we have to acknowledge. If someone should get onto my machine while it's backing up, they could see the whole command line passed to the Truecrypt process and thus be able to get it that way. Again, at that point my machine's been compromised while I'm logged in with the home directory decrypted. I'm done no matter what. Since the plan is for this to be plugged in once a week for only a few minutes, I'm not as concerned about this particular security issue.
So, this script mounts the encrypted partition we created before and then wipes history to make sure there isn't a trail with my password, just in case. When I started playing around with this, I thought I'd be able to just use the password option on the command line, but I found it absolutely would not work unless I put all of the other options in, including using a keyfile. I don't know why that is, nor do I feel like digging far to find out. The remaining two options included:
--protect-hidden=no tells Truecrypt that there's no hidden partition it needs to be concerned about
--fs-options="uid=1000,gid=1000" tells it to set the newly mounted volume as owned by my user account so I can work with the files.
We need another command to unmount the disk when backup's done. As mentioned above, I'm not adding my personal .bin directory to my path until I'm sure I need it, and the command to unmount is only one line, so I decide instead to create a Bash alias. Append the following to the end of your .bash_aliases file:
alias urose="truecrypt -d /mnt/rosewill"
Now we have the mounting and unmounting of the disk automated. Of course, being the lazy fellow I am, I certainly don't want to have to TYPE these commands when I want to back up. I want it to do even that trivial task for me. S'okay. Computers are here to do our work for us, remember? Besides, having the computer do this means I literally have to remember to plug the thing in once a week. Hmmmm...I think I just came up with an excuse to build a robot!
Back to the matter...how do we automate this process? Fortunately, this part's trivial. We write what's known as a udev rule. udev is the Linux subsystem that detects hardware and creates device nodes for them. When you plug in a device hot, udev reads its pre-defined rules to determine what to do with it. We're just adding one of our own that's particular to this single, unique device. First, we find out the perts* about the drive by plugging it in, turning it on and using udevadm to find out more info:
udevadm info -a --name=/dev/sdb
This will give you TONS of useful information about your drive, but we only need a little bit. Specifically, I need to know the manufacturer and model number of what just got plugged in so I ensure it's the disk I want and not some USB flash drive or something. Now, you'll notice this info is broken up into sections that begin "looking at parent device". This is because udevadm looks at the device and then everything that links this device to the OS. Since this is a USB enclosure, that means: the IDE-USB interface, the USB interface, the USB hub, the USB-PCI bridge, etc, etc. I'm most interested in the section below. Keep in mind, when you're writing udev rules, the attributes you're looking at must all reside in one of these blocks. You can't pull the vendor attribute from the SUBSYSTEMS="usb" section and the model from the SUBSYSTEMS="scsi" one. The reason I chose the block below is because that block had all of the relevant data I need.
looking at parent device '/devices/pci0000:00/0000:00:0b.1/usb1/1-1/1-1:1.0/host6/target6:0:0/6:0:0:0':
KERNELS=="6:0:0:0"
SUBSYSTEMS=="scsi"
DRIVERS=="sd"
ATTRS{device_blocked}=="0"
ATTRS{type}=="0"
ATTRS{scsi_level}=="0"
ATTRS{vendor}=="Initio "
ATTRS{model}=="WD2000JB-00EVA0 "
ATTRS{rev}=="1.06"
ATTRS{state}=="running"
ATTRS{timeout}=="30"
ATTRS{iocounterbits}=="32"
ATTRS{iorequest_cnt}=="0x48"
ATTRS{iodone_cnt}=="0x48"
ATTRS{ioerr_cnt}=="0x1"
ATTRS{modalias}=="scsi:t-0x00"
ATTRS{evt_media_change}=="0"
ATTRS{dh_state}=="detached"
ATTRS{queue_depth}=="1"
ATTRS{queue_type}=="none"
ATTRS{max_sectors}=="240"
We now have all we need to create a rule. First, su and joe /etc/udev/rules.d/10-local.rules. We only need one line for this:
KERNEL=="sd*", SUBSYSTEMS=="scsi", ATTRS{vendor}=="Initio", ATTRS{model}=="WD2000JB-00EVA0", SYMLINK+="rosewill%n", RUN+="/bin/sh /home/tonyk/.bin/mountrose"
What that's saying is "when you see a device that the kernel labels a scsi disk, from the vendor Initio with this model number symlink its devnode as rosewill. Also, since this is a disk, symlink the devnodes for each partition as rosewill1, rosewill2, etc and then run the script /home/akarakas/.bin/mountrose using /bin/sh".
So, why do we do it like this? Let's say, for example, I've plugged in a USB flash drive prior to plugging in this hard drive. There's a chance that flash drive might become /dev/sdb and my backup hard drive /dev/sdc. Can't really write a script for variable disk names, now can I? By setting up the udev rule in this way, this disk will ALWAYS be accessible via /dev/rosewill and its partitions via /dev/rosewillX. I don't need to poke and prod as part of my script, I'm always going to have the right devnode. As to why I launch the script this way, for some reason a bash script doesn't honor the shebang properly when run by udev. I could not find any way to run this script by simply calling it. No biggie, it works.
And now, the moment of truth...plug it in, turn it on and...../mnt/rosewill is mounted, and I can see and modify the files. Woo hoo! Now we have my media setup! Kewl!
At this point, I'm going to conclude this article and leave the remainder for Part 2. In that article, I'll present the full script the system runs when the disk is plugged in and reasons for some of the choices I've made.
*perts = Pertinent details. I'm trying it out, what do you think?
Sunday, October 24, 2010
Console-ing myself transparently
I spend a lot of time in the command line, so it would be convenient for me to have access to a console at all times. The only problem is it's so easy to forget not close it, plus it's never big enough, plus I don't want it in the task list/pager, etc, etc. I've tried solutions like Yakuake and Guake, but they never quite fit my needs.
Besides, I'm playing around with Linux, right? Might as well try cobbling together a Linuxish solution that works exactly how *I* want it to. So, first, I put together some requirements:
- Needs to always be available.
- Needs to be unobtrusive.
- Needs to be big enough to see full lines of whatever I was working on (such as looking at logs).
- Needs to not hide the desktop.
After looking over my requirements, I decided that what I was looking for was essentially one of those cool transparent and undecorated consoles that you see in the screenshots of the really cool kids. First, we need to install a couple of packages: sudo aptitude install wmctrl devilspie. I'll explain their uses as I get to them.
The first one we'll setup is Devil's Pie. This is a unique utility that runs in the background. At launch, it reads in any Devli's Pie scripts in ~/.devilspie. These scripts are made up of rules that tell Devil's Pie "when you see a window launch that matches these attributes, apply these settings to it". You can set placement (such as making sure an app always launches in a particular location or workspace) or display characteristics. We're going to use the latter to remove the title bar from a very specific console and nail it to the desktop.
To start, open Gnome Terminal (you can use whatever you want, just apply similar settings). Edit -> Profiles -> New.
- On the General tab, give the profile a name. I'm calling mine DesktopConsole. Uncheck "Show menubar by default in new terminals."
- Title and Command tab, set the Initial Title also to DesktopConsole and set "When terminal commands set their own titles" to "Keep initial title".
- Background tab, uncheck "Use background settings from system theme" and choose Transparent background. Set transparency to whatever's most comfortable for you. We'll be modifying this on the fly later, but need an initial setting.
- Scrolling tab, Scrollbar is Disabled.
Apply any other customizations, such as colors, as you like and then save the profile. Now, to make sure it launches at startup, we can just add it to Startup Applications in System -> Preferences, but I have need of a little more control, so we're going to create a script to launch it with. I put all of my scripts in ~/.bin, so
joe ~/.bin/launchconsole
Into which I add:
This launches a new terminal window that uses my custom profile and launches Gnu screen. Screen's a tool that's well worth looking in to, especially for a setup like this. It allows you to essentially have multiple terminal sessions to use as you need. We will be going into more detail about it in later articles. We'll also see a little later why I used a script to launch a single command. Hint: it's not going to be alone for long! And, though I'm spreading this console across the desktop, I still specified a geometry. For some reason, what we do next never works properly without it.
If we launched it now, we'd get a transparent console, yes, but it's free-floating and still has a titlebar. Time for a slice of the Devil's Pie. If you don't already have one, you need to create a directory in your home for the scripts (mkdir ~/.devilspie). Then, we create the script to do all of the magic...or at least most of it (joe ~/.devilspie/DesktopConsole.ds)
Each Devil's Pie script can contain only one rule. In this case, the rule is "if you see a window with the name DesktopConsole apply these properties to it":
- undecorate: Removes the title bar from the window.
- skip_pager: Prevents the window from appearing in your alt-tab list of windows.
- skip_tasklist: Prevents it from showing up on taskbars.
- wintype "desktop": I played around with quite a few different types before finally finding and settling on "desktop". When you set this property, the window manager thinks of the window as another desktop. That means you can do all of the "show desktop" things (which minimize all windows) and the console remains. It also means the console is always below other windows.
So, to verify it works, we log off and back on and are presented with our lovely desktop console:
A little small, though, huh? Let's fix that by modifying that launchconsole script:
#!/bin/sh
# First we need to know the screen's resolution
screenResolution="`xrandr | grep \* | cut -d' ' -f4`"
# The above command returns the geometry of the screen, i.e. 1600x900
# So, we need to split that into two numbers we can plug in elsewhere in the script
# In the next couple of lines, we're using the bash split function and putting each
# number into its own cell in an array.
IFS="x"
arr=( $screenResolution )
IFS=""
# echo ${arr[1]}
screenHeight=${arr[1]}
screenWidth=${arr[0]}
# My top panel is 32 pixels high. The below setting ensures the console fills the
# remaining desktop
consoleFullHeight=$(($screenHeight-33))
# Launch the terminal
gnome-terminal --window-with-profile=DesktopConsole --command=screen
# wmctrl is a utility that allows you to control windows and their placement from the
# command line. This command tells it to place the console starting 33 pixels down
# from the top-left corner (right below the panel), and fill the screen.
wmctrl -r "DesktopConsole" -e 0,0,33,$screenWidth,$consoleFullHeight
# As you'll see in later articles, I've got some plans in mind to make this console
# usable in a Yakuake/Guake kind of way. As part of those plans, I'll need to modify the
# transparency of the console as needed. In order to ensure we start off
# transparent enough, we'll set this in the initial launch script.
gconftool --set "/apps/gnome-terminal/profiles/Profile0/background_darkness" --type float .01
So, now we save this file, log off and log back on again.
Much better this time. We're good with this for the moment, but I've got an addition in the works to share with you soon. Beyond that, I've got my screen config and what I use that for. Hint: gMail and Mutt. gMutt.
Besides, I'm playing around with Linux, right? Might as well try cobbling together a Linuxish solution that works exactly how *I* want it to. So, first, I put together some requirements:
- Needs to always be available.
- Needs to be unobtrusive.
- Needs to be big enough to see full lines of whatever I was working on (such as looking at logs).
- Needs to not hide the desktop.
After looking over my requirements, I decided that what I was looking for was essentially one of those cool transparent and undecorated consoles that you see in the screenshots of the really cool kids. First, we need to install a couple of packages: sudo aptitude install wmctrl devilspie. I'll explain their uses as I get to them.
The first one we'll setup is Devil's Pie. This is a unique utility that runs in the background. At launch, it reads in any Devli's Pie scripts in ~/.devilspie. These scripts are made up of rules that tell Devil's Pie "when you see a window launch that matches these attributes, apply these settings to it". You can set placement (such as making sure an app always launches in a particular location or workspace) or display characteristics. We're going to use the latter to remove the title bar from a very specific console and nail it to the desktop.
To start, open Gnome Terminal (you can use whatever you want, just apply similar settings). Edit -> Profiles -> New.
- On the General tab, give the profile a name. I'm calling mine DesktopConsole. Uncheck "Show menubar by default in new terminals."
- Title and Command tab, set the Initial Title also to DesktopConsole and set "When terminal commands set their own titles" to "Keep initial title".
- Background tab, uncheck "Use background settings from system theme" and choose Transparent background. Set transparency to whatever's most comfortable for you. We'll be modifying this on the fly later, but need an initial setting.
- Scrolling tab, Scrollbar is Disabled.
Apply any other customizations, such as colors, as you like and then save the profile. Now, to make sure it launches at startup, we can just add it to Startup Applications in System -> Preferences, but I have need of a little more control, so we're going to create a script to launch it with. I put all of my scripts in ~/.bin, so
joe ~/.bin/launchconsole
Into which I add:
#!/bin/sh
gnome-terminal --window-with-profile=DesktopConsole --command=screen --geometry=400x200This launches a new terminal window that uses my custom profile and launches Gnu screen. Screen's a tool that's well worth looking in to, especially for a setup like this. It allows you to essentially have multiple terminal sessions to use as you need. We will be going into more detail about it in later articles. We'll also see a little later why I used a script to launch a single command. Hint: it's not going to be alone for long! And, though I'm spreading this console across the desktop, I still specified a geometry. For some reason, what we do next never works properly without it.
If we launched it now, we'd get a transparent console, yes, but it's free-floating and still has a titlebar. Time for a slice of the Devil's Pie. If you don't already have one, you need to create a directory in your home for the scripts (mkdir ~/.devilspie). Then, we create the script to do all of the magic...or at least most of it (joe ~/.devilspie/DesktopConsole.ds)
(if
(matches (window_name) "DesktopConsole")
(begin
(undecorate)
(skip_pager)
(skip_tasklist)
(wintype "desktop")
)
)Each Devil's Pie script can contain only one rule. In this case, the rule is "if you see a window with the name DesktopConsole apply these properties to it":
- undecorate: Removes the title bar from the window.
- skip_pager: Prevents the window from appearing in your alt-tab list of windows.
- skip_tasklist: Prevents it from showing up on taskbars.
- wintype "desktop": I played around with quite a few different types before finally finding and settling on "desktop". When you set this property, the window manager thinks of the window as another desktop. That means you can do all of the "show desktop" things (which minimize all windows) and the console remains. It also means the console is always below other windows.
So, to verify it works, we log off and back on and are presented with our lovely desktop console:
A little small, though, huh? Let's fix that by modifying that launchconsole script:
#!/bin/sh
# First we need to know the screen's resolution
screenResolution="`xrandr | grep \* | cut -d' ' -f4`"
# The above command returns the geometry of the screen, i.e. 1600x900
# So, we need to split that into two numbers we can plug in elsewhere in the script
# In the next couple of lines, we're using the bash split function and putting each
# number into its own cell in an array.
IFS="x"
arr=( $screenResolution )
IFS=""
# echo ${arr[1]}
screenHeight=${arr[1]}
screenWidth=${arr[0]}
# My top panel is 32 pixels high. The below setting ensures the console fills the
# remaining desktop
consoleFullHeight=$(($screenHeight-33))
# Launch the terminal
gnome-terminal --window-with-profile=DesktopConsole --command=screen
# wmctrl is a utility that allows you to control windows and their placement from the
# command line. This command tells it to place the console starting 33 pixels down
# from the top-left corner (right below the panel), and fill the screen.
wmctrl -r "DesktopConsole" -e 0,0,33,$screenWidth,$consoleFullHeight
# As you'll see in later articles, I've got some plans in mind to make this console
# usable in a Yakuake/Guake kind of way. As part of those plans, I'll need to modify the
# transparency of the console as needed. In order to ensure we start off
# transparent enough, we'll set this in the initial launch script.
gconftool --set "/apps/gnome-terminal/profiles/Profile0/background_darkness" --type float .01
So, now we save this file, log off and log back on again.
Much better this time. We're good with this for the moment, but I've got an addition in the works to share with you soon. Beyond that, I've got my screen config and what I use that for. Hint: gMail and Mutt. gMutt.
Thursday, October 14, 2010
My semi-regular trip to the dark side
Windows, Windows, Windows...I do love me some Windows! In 1994, after recently upgrading to Windows for Workgroups 3.11, I had a revelation. These were the early days of the Internet. There was no web, you used Gopher instead. There were no forums, we used Usenet and we LIKED IT! Broadband? Hell, dial-up networking? What are these? You wanted to get on the Internet, you used something like Telix* and communicated with the world via a command line. It was a wild time.
Where was I? Oh, yes...anyway, one night I needed to download some massive 1 or 2 megabyte files over my 19k modem from school, but I also wanted to play this new game I'd just bought called Warcraft (no "World of", just Warcraft). What was I to do? I thought about risking running both Telix and Warcraft under WfW and hope for the best. After all, this new WinG platform Microsoft was touting was supposed to make everything work smooth, right?
So, I went for it. I got thoroughly immersed in the game for a while and lost track of time. Not thinking, I Alt-Tabbed over to Telix to see how things were going and immediately thought "Oh, crap! There goes everything I was doing!" You see, back in those days, Windows had a bit of a reputation. If you tried to run more than one thing at a time, it would crash! You certainly didn't try multi-tasking a DOS app with anything, and certainly not with a game! "You're a fool!" the crowds would shout!
But, to my surprise, the downloads were still progressing. I was still online with Telix! "Wow" I thought. "That's weird." But, I just KNEW my game would be hosed. So, I Alt-Tabbed back to see the carnage....and my little peons that I had told just prior to switching to go gather some trees were just heading out to begin a-choppin'. The game hadn't frozen...it had paused! This made no sense...
That's when the revelation hit me: this was a fresh install of Windows for Workgroups 3.11. That means it was at least a pretty good approximation of what Microsoft intended your experience to be and it worked...as long as I hadn't screwed around with it. The revelation was disheartening: PEBKAC. Like most people, I blamed Microsoft for all of my woes, but the reality is an operating system is only as stable as its admin.
Having taken the "It's Windows, what do you expect" hammer out of my toolbox, I started to really get an appreciation for Microsoft's products. Now, I know when something's broke it can be fixed. A person doesn't have to live with blue screens. They are not, as one commenter once said "Microsoft's way of telling you you need to reboot". They're error messages that contain valuable information about what's wrong!
I attest my success in the IT field to the way I approach things. For example, I follow Microsoft's best practices as closely as possible because that's what they tested! They know THAT configuration works and this one doesn't. You can't support a half billion dollar/year revenue generating environment on an unstable and unreliable platform...and that's why we run ours on Windows! But, I'm not here to evangelize Microsoft products to you or try to convince you that if you have negative impressions of them the fault is yours alone (even though that's true :) ). No, I'm here today to tell you about my current interest episode which includes Ubuntu Linux.
"LINUX? You think I'm going to listen to some WINDOWS guy badmouth my beloved OS?"
Nope. You see, I mentioned earlier that my install of WfW was a fresh one. Why do you think it was fresh? Because I had been giving this new OS called "Linux" a trial run on my machine. I'd been reading about it for a while on the Intertubes about how light and fast and easy on resources it was. As I was a Computer Science major at the University of Rochester, I wanted to be one of the cool kids using Unix so I set out trying to find it. I actually headed to the UR computing center one day with a box of 4-5 floppy disks to see if someone could put it on there for me. Hey, I was told it was small and light, and since DOS + Windows amounted to 6-7 disks, Linux must be MUCH less, right?
After poking around, I managed to find a copy of the only distribution in town: Slackware. It wasn't even Slackware 1.0 since the Linux kernel itself was still only around version 0.98, but I downloaded all 15 disks (15!!), wiped my computer and installed it. And, it didn't work...Well, let me clarify. The OS worked, but the X-Windows system that provided a GUI on top of it didn't work all that well. I had a crappy Trident video card that could only do 640x400 (that's not a typo. It wasn't 480, it was 400) in 16 colors and X didn't know how to do that well. There also weren't very many X-based programs out there for it. Basically, you could multitask some console windows, have a nifty clock window, and a pair of eyes that followed your mouse around. Very unimpressive. Click the link for X-Windows. The Wikipedia article has a picture of about the best you could hope for back then at the top. Ugly, right?
But, I used it for a while anyway. I struggled, I fought and eventually...I reinstalled Windows. There was no benefit whatsoever to me running this OS at the time. In fact, thanks to my crappy video card it was much crashier than Windows! But, I remained interested in it. I followed development over the years. When the kernel hit 1.0, I purchased a new machine to be my primary and installed it on the old (after upgrading that video card, of course). Each year since I'd give the latest and greatest a test run here and there. Sometimes they'd last days, sometimes months. Sometimes it never made it past being installed in a virtual machine.
For a good long time, I even used Linux From Scratch as my primary OS. In fact, I spoke at VPNCon in Toronto and Linux World Expo in Boston about using Linux From Scratch to provide my company at the time with a free VPN solution that worked wonders! Many an article (halfway down, ignore the horrible picture) was written about the one trick this pony pulled off there.
But, while LFS was fun, it was a pain in the ass. Each upgrade required careful tweaking and tuning to make sure I didn't break a hundred other things. Some things never worked right, and others just never happened at all. I became a believer in package management systems. Redhat's yum made me happy for a while, but it was eventually trying out Debian (pronounced DEB-ian, not DE-bian. It's named after the original author, Ian and his girlfriend, Deb. Betcha didn't know that!) that gave me an inkling that this "year of the Linux desktop" I'd been hearing about for a decade or so might come to pass.
A few years back a new distribution that was based on Debian called Ubuntu hit the scene and really made a splash. Ubuntu was designed to combine the ease of upgrading/installing/maintaining from a Debian system with the usability of a Redhat desktop. They've got some good directives and history behind them, not to mention a huge fan base. The most recent iterations, 10.04 and as of last Sunday 10.10, have been widely acclaimed as "Windows-killers". My wife's actually been using a Xubuntu desktop for a couple of years now, and aside with some weirdness with her trackpad I can't replicate, I've gotten no complaints from her.
And, the point of all this? About three weeks ago, I wiped my little laptop that was running Windows 7 (I really do love me some Windows 7, though!) and put the beta of Xubuntu 10.10 on. I ran it for about two weeks during which time I installed the Netbook Remix desktop, the Ubuntu desktop, Lubuntu desktop and Edubuntu desktop on top of it....and seriously borked the install. But, that's ok. I was really putting it through its paces to see which install I wanted to use as a base. I had partitioned my drive up with a 10G OS partition and the remainder setup as home, so when I finally decided to just go with the plain vanilla Ubuntu it was a 20 minute reinstall.
I did have some issues with my wireless for a while there, but I was able to work those out and have gotten down to the business of serious fiddling. Yeah, Ubuntu's got a reputation in the community as "the Windows of Linux" because it's easy to use. The gods forbid! But, this is my primary machine. This is what I use to do...whatever it is I do. It needs to work! Playing is fine. Fiddling is fine, but at the end of the day when I need to do something it better be able to do with it! And, that's really the point: I have an OS that I can use AND play with. It's still Linux under there, and once you see what I've done to it under the hood, you'll see that.
So, the articles I have on tap detail the things I've done as well as possible plans for the future. Could be fun!
* Seriously? You're still SELLING Telix? For $79.99? You DO know DOS is kinda over, right? Don't get me wrong, I LOVED Telix! It was one of the most popular pieces of shareware ever and with good reason: it worked and worked well. It was one of the few I remember actually paying for! But...do you really still have people paying for it?
Where was I? Oh, yes...anyway, one night I needed to download some massive 1 or 2 megabyte files over my 19k modem from school, but I also wanted to play this new game I'd just bought called Warcraft (no "World of", just Warcraft). What was I to do? I thought about risking running both Telix and Warcraft under WfW and hope for the best. After all, this new WinG platform Microsoft was touting was supposed to make everything work smooth, right?
So, I went for it. I got thoroughly immersed in the game for a while and lost track of time. Not thinking, I Alt-Tabbed over to Telix to see how things were going and immediately thought "Oh, crap! There goes everything I was doing!" You see, back in those days, Windows had a bit of a reputation. If you tried to run more than one thing at a time, it would crash! You certainly didn't try multi-tasking a DOS app with anything, and certainly not with a game! "You're a fool!" the crowds would shout!
But, to my surprise, the downloads were still progressing. I was still online with Telix! "Wow" I thought. "That's weird." But, I just KNEW my game would be hosed. So, I Alt-Tabbed back to see the carnage....and my little peons that I had told just prior to switching to go gather some trees were just heading out to begin a-choppin'. The game hadn't frozen...it had paused! This made no sense...
That's when the revelation hit me: this was a fresh install of Windows for Workgroups 3.11. That means it was at least a pretty good approximation of what Microsoft intended your experience to be and it worked...as long as I hadn't screwed around with it. The revelation was disheartening: PEBKAC. Like most people, I blamed Microsoft for all of my woes, but the reality is an operating system is only as stable as its admin.
Having taken the "It's Windows, what do you expect" hammer out of my toolbox, I started to really get an appreciation for Microsoft's products. Now, I know when something's broke it can be fixed. A person doesn't have to live with blue screens. They are not, as one commenter once said "Microsoft's way of telling you you need to reboot". They're error messages that contain valuable information about what's wrong!
I attest my success in the IT field to the way I approach things. For example, I follow Microsoft's best practices as closely as possible because that's what they tested! They know THAT configuration works and this one doesn't. You can't support a half billion dollar/year revenue generating environment on an unstable and unreliable platform...and that's why we run ours on Windows! But, I'm not here to evangelize Microsoft products to you or try to convince you that if you have negative impressions of them the fault is yours alone (even though that's true :) ). No, I'm here today to tell you about my current interest episode which includes Ubuntu Linux.
"LINUX? You think I'm going to listen to some WINDOWS guy badmouth my beloved OS?"
Nope. You see, I mentioned earlier that my install of WfW was a fresh one. Why do you think it was fresh? Because I had been giving this new OS called "Linux" a trial run on my machine. I'd been reading about it for a while on the Intertubes about how light and fast and easy on resources it was. As I was a Computer Science major at the University of Rochester, I wanted to be one of the cool kids using Unix so I set out trying to find it. I actually headed to the UR computing center one day with a box of 4-5 floppy disks to see if someone could put it on there for me. Hey, I was told it was small and light, and since DOS + Windows amounted to 6-7 disks, Linux must be MUCH less, right?
After poking around, I managed to find a copy of the only distribution in town: Slackware. It wasn't even Slackware 1.0 since the Linux kernel itself was still only around version 0.98, but I downloaded all 15 disks (15!!), wiped my computer and installed it. And, it didn't work...Well, let me clarify. The OS worked, but the X-Windows system that provided a GUI on top of it didn't work all that well. I had a crappy Trident video card that could only do 640x400 (that's not a typo. It wasn't 480, it was 400) in 16 colors and X didn't know how to do that well. There also weren't very many X-based programs out there for it. Basically, you could multitask some console windows, have a nifty clock window, and a pair of eyes that followed your mouse around. Very unimpressive. Click the link for X-Windows. The Wikipedia article has a picture of about the best you could hope for back then at the top. Ugly, right?
But, I used it for a while anyway. I struggled, I fought and eventually...I reinstalled Windows. There was no benefit whatsoever to me running this OS at the time. In fact, thanks to my crappy video card it was much crashier than Windows! But, I remained interested in it. I followed development over the years. When the kernel hit 1.0, I purchased a new machine to be my primary and installed it on the old (after upgrading that video card, of course). Each year since I'd give the latest and greatest a test run here and there. Sometimes they'd last days, sometimes months. Sometimes it never made it past being installed in a virtual machine.
For a good long time, I even used Linux From Scratch as my primary OS. In fact, I spoke at VPNCon in Toronto and Linux World Expo in Boston about using Linux From Scratch to provide my company at the time with a free VPN solution that worked wonders! Many an article (halfway down, ignore the horrible picture) was written about the one trick this pony pulled off there.
But, while LFS was fun, it was a pain in the ass. Each upgrade required careful tweaking and tuning to make sure I didn't break a hundred other things. Some things never worked right, and others just never happened at all. I became a believer in package management systems. Redhat's yum made me happy for a while, but it was eventually trying out Debian (pronounced DEB-ian, not DE-bian. It's named after the original author, Ian and his girlfriend, Deb. Betcha didn't know that!) that gave me an inkling that this "year of the Linux desktop" I'd been hearing about for a decade or so might come to pass.
A few years back a new distribution that was based on Debian called Ubuntu hit the scene and really made a splash. Ubuntu was designed to combine the ease of upgrading/installing/maintaining from a Debian system with the usability of a Redhat desktop. They've got some good directives and history behind them, not to mention a huge fan base. The most recent iterations, 10.04 and as of last Sunday 10.10, have been widely acclaimed as "Windows-killers". My wife's actually been using a Xubuntu desktop for a couple of years now, and aside with some weirdness with her trackpad I can't replicate, I've gotten no complaints from her.
And, the point of all this? About three weeks ago, I wiped my little laptop that was running Windows 7 (I really do love me some Windows 7, though!) and put the beta of Xubuntu 10.10 on. I ran it for about two weeks during which time I installed the Netbook Remix desktop, the Ubuntu desktop, Lubuntu desktop and Edubuntu desktop on top of it....and seriously borked the install. But, that's ok. I was really putting it through its paces to see which install I wanted to use as a base. I had partitioned my drive up with a 10G OS partition and the remainder setup as home, so when I finally decided to just go with the plain vanilla Ubuntu it was a 20 minute reinstall.
I did have some issues with my wireless for a while there, but I was able to work those out and have gotten down to the business of serious fiddling. Yeah, Ubuntu's got a reputation in the community as "the Windows of Linux" because it's easy to use. The gods forbid! But, this is my primary machine. This is what I use to do...whatever it is I do. It needs to work! Playing is fine. Fiddling is fine, but at the end of the day when I need to do something it better be able to do with it! And, that's really the point: I have an OS that I can use AND play with. It's still Linux under there, and once you see what I've done to it under the hood, you'll see that.
So, the articles I have on tap detail the things I've done as well as possible plans for the future. Could be fun!
* Seriously? You're still SELLING Telix? For $79.99? You DO know DOS is kinda over, right? Don't get me wrong, I LOVED Telix! It was one of the most popular pieces of shareware ever and with good reason: it worked and worked well. It was one of the few I remember actually paying for! But...do you really still have people paying for it?
Thursday, April 22, 2010
Maximizing storage in the cloud
It's been a while since I've written something for this blog. That's primarily because of a change in jobs. My old job was as a Windows Server Engineer. I fixed stuff, so I encountered weird problems from time to time and liked sharing that info. The new position is as Situational Lead. My company has adopted ITIL protocols and my new job concerns incident management. In a nutshell, when there's a big outage, it's my job to gather the team together who will get it fixed and manage the incident until service is restored. I don't fix much anymore.
So, that means I won't be talking as much about technical stuff, I have other ideas in mind. But, for now, I wanted to talk a little bit about some services that have been around for some time now that I'm just getting around to exploring: cloud storage. Like a lot of people these days, I've got a USB flash drive I carry everywhere. Actually, I've got two: a 4G one and a 1Gigger. Sometimes it's very useful to get files back and forth to the office, and it's a whole lot easier carrying a USB stick than taking my whole laptop home with me. The big problem, though, is making sure the files you need on there actually ARE on there. Recently, my company has implemented desktop drive encryption, with it being forced for removable drives. If I want to copy from my computer to the USB drive, it has to be encrypted (there's no such restriction for USB -> Computer transfers).
Recently, my friends and I have started up a PBeM game of Trail of Cthulhu. For a while, I had been storing the rule books, which are in digital format, in my e-mail, but it started to become cumbersome with the number of files I had to maintain, plus notes files, etc. It was looking like a flash drive was the way to go when I remembered about companies like Dropbox, iDriveSync and Box.net. For a reasonable monthly fee, these companies will give you fairly large amounts of storage space that you can use however you want. For free, the first two give you 2G and Box.net gives you one. 2G is a nice chunk of space, so I decided to sign up for Dropbox and see if I might get some use out of it.
That was last Friday. Since then I've used the hell out of these things! Firstly, the first two offer extensions you can install on your computer that will match an ordinary folder on your computer with the online storage. This means managing files is as easy as it already is in Explorer. Drag files into the special folder, they're automatically synced with your storage. If you've got the client installed on multiple machines, all of them are synced simultaneously. Now, if I need to get a file to the office, I just drop it into my Dropbox folder at home and it'll be on my office PC before I get there.
But, the real beauty of these services lies beyond their ability to assist you in moving files. They're a great way of getting large files to other people. I had a digital copy of a video a friend of mine wanted to see, but it was way too big to send via e-mail. No problem, I copied it into to my Dropbox folder. I right-clicked the file in Explorer and selected "Copy Public URL". This placed in my clipboard a URL I could send my friend who could then just click the link and download the file. No fuss, no muss, no setting things up. And, since the file was downloading from Dropbox' servers, it was fast! A whole lot faster than if I'd tried to setup an FTP site on my home computer or something.
You get all of that, plus you can access those files yourself at any time, from any computer, by just logging into the website. Keep a copy of an anti-virus program you like to have available for when you go to a friend's house, but forgot your USB drive? No problem. I really have gotten a lot of use out of these services in just the last week alone, and highly recommend them to anyone who works with files in multiple locations. A few tips, though:
- Dropbox has a referral program. If you click the link in the preceding sentence, you'll be able to sign up for the service using me as a referral. Doing so means you and I will each get an extra 250M of storage space. It's not much, but you can go up to a full 8G of free additional space if you get enough referrals. That's not too shabby!
- I'm not a fan of having personal stuff on my work computer. That's not to say I don't, just that I don't like it. At any time, my computer could get audited, and I don't think I'd get a lot of crap for having Trail of Cthulhu rulebooks on it, but why take that chance? What I did was installed Truecrypt. I then created a 2G volume and pointed Dropbox at that. Now, whatever is in my Dropbox is encrypted when on my work PC. This has the added benefit in that when mounted, the volume shows up as a drive. All I have to do is look at how much free space is available on that drive and I know instantly how much free space I have in Dropbox. Of course, I now have a couple of referrals, so I'll have to create a new volume to match the size.
- Within my Dropbox, I've got a folder called "Move Off". I have a script that runs once an hour on my machine at home that monitors that folder. If something shows up in it, it's moved off into another location. I use my Dropbox for non-private, but still personal data. If, for example, I want to get the pictures off my iPhone while in the office, I can do so. I just put them into Move Off and they're whisked away soon after.
These companies provide a great service to you and I and I highly recommend checking them out. I'm even considering paying for one of them, once I've decided who gives most consistently high level of availability! Oh, and did I mention that some have iPhone apps that allow you to view and manage your files on the road, too?
So, that means I won't be talking as much about technical stuff, I have other ideas in mind. But, for now, I wanted to talk a little bit about some services that have been around for some time now that I'm just getting around to exploring: cloud storage. Like a lot of people these days, I've got a USB flash drive I carry everywhere. Actually, I've got two: a 4G one and a 1Gigger. Sometimes it's very useful to get files back and forth to the office, and it's a whole lot easier carrying a USB stick than taking my whole laptop home with me. The big problem, though, is making sure the files you need on there actually ARE on there. Recently, my company has implemented desktop drive encryption, with it being forced for removable drives. If I want to copy from my computer to the USB drive, it has to be encrypted (there's no such restriction for USB -> Computer transfers).
Recently, my friends and I have started up a PBeM game of Trail of Cthulhu. For a while, I had been storing the rule books, which are in digital format, in my e-mail, but it started to become cumbersome with the number of files I had to maintain, plus notes files, etc. It was looking like a flash drive was the way to go when I remembered about companies like Dropbox, iDriveSync and Box.net. For a reasonable monthly fee, these companies will give you fairly large amounts of storage space that you can use however you want. For free, the first two give you 2G and Box.net gives you one. 2G is a nice chunk of space, so I decided to sign up for Dropbox and see if I might get some use out of it.
That was last Friday. Since then I've used the hell out of these things! Firstly, the first two offer extensions you can install on your computer that will match an ordinary folder on your computer with the online storage. This means managing files is as easy as it already is in Explorer. Drag files into the special folder, they're automatically synced with your storage. If you've got the client installed on multiple machines, all of them are synced simultaneously. Now, if I need to get a file to the office, I just drop it into my Dropbox folder at home and it'll be on my office PC before I get there.
But, the real beauty of these services lies beyond their ability to assist you in moving files. They're a great way of getting large files to other people. I had a digital copy of a video a friend of mine wanted to see, but it was way too big to send via e-mail. No problem, I copied it into to my Dropbox folder. I right-clicked the file in Explorer and selected "Copy Public URL". This placed in my clipboard a URL I could send my friend who could then just click the link and download the file. No fuss, no muss, no setting things up. And, since the file was downloading from Dropbox' servers, it was fast! A whole lot faster than if I'd tried to setup an FTP site on my home computer or something.
You get all of that, plus you can access those files yourself at any time, from any computer, by just logging into the website. Keep a copy of an anti-virus program you like to have available for when you go to a friend's house, but forgot your USB drive? No problem. I really have gotten a lot of use out of these services in just the last week alone, and highly recommend them to anyone who works with files in multiple locations. A few tips, though:
- Dropbox has a referral program. If you click the link in the preceding sentence, you'll be able to sign up for the service using me as a referral. Doing so means you and I will each get an extra 250M of storage space. It's not much, but you can go up to a full 8G of free additional space if you get enough referrals. That's not too shabby!
- I'm not a fan of having personal stuff on my work computer. That's not to say I don't, just that I don't like it. At any time, my computer could get audited, and I don't think I'd get a lot of crap for having Trail of Cthulhu rulebooks on it, but why take that chance? What I did was installed Truecrypt. I then created a 2G volume and pointed Dropbox at that. Now, whatever is in my Dropbox is encrypted when on my work PC. This has the added benefit in that when mounted, the volume shows up as a drive. All I have to do is look at how much free space is available on that drive and I know instantly how much free space I have in Dropbox. Of course, I now have a couple of referrals, so I'll have to create a new volume to match the size.
- Within my Dropbox, I've got a folder called "Move Off". I have a script that runs once an hour on my machine at home that monitors that folder. If something shows up in it, it's moved off into another location. I use my Dropbox for non-private, but still personal data. If, for example, I want to get the pictures off my iPhone while in the office, I can do so. I just put them into Move Off and they're whisked away soon after.
These companies provide a great service to you and I and I highly recommend checking them out. I'm even considering paying for one of them, once I've decided who gives most consistently high level of availability! Oh, and did I mention that some have iPhone apps that allow you to view and manage your files on the road, too?
Friday, January 8, 2010
Resolving difficulties with using certs
Recently, my company has begun working on integrating the various services and products we offer. Since most of our customers purchase or use multiple services, it only makes sense that the data they have in one should be available in all (I know, sometimes the obvious takes a while). To do this, we're using .Net web services to allow this to happen. In order to protect these services, we're using code signing certificates and access gateways that verify these certs.
A problem we've run into is that most of our internal users in these environments connect via Citrix Xenapp servers. In order to limit the management of certs in this environment, the code signing certs are added to the local machine, rather than individual, stores. This allows any user of these machines to connect to the services without our having to make sure the certs are installed in everyone's profiles, etc. We had a release that was going live today that served as integration between two services, but found they couldn't talk to each other. It turns out that access to certs was the issue.
When installing these certs, we simply installed them by double-clicking them and following the import wizard. We'd tell the wizard to install them into the local machine store and then used the winhttpcertcfg utility that comes with Windows to grant only specific groups access to these certs. When put into place, though, on our test systems we found that no users could use the service. When we checked permissions on these certs, the users definitely had been granted access to them, but it just wasn't working.
When we had users log into machines that had the cert and asked them to run the command:
winhttpcertcfg -l -c LOCAL_MACHINE\TrustedPeople -s certname
They would get the following error:
Error: Access was not successfully obtained for the private key. This
can only be done by the user who installed the certificate.
When an admin, namely myself, ran the same command we'd see that the users had been explicitly granted access. This made no sense whatsoever, and in googling the error I found I wasn't alone in my confusion. During this search, I came across a utility that's put out by Microsoft called the FindPrivateKey tool. After using this spiffy little tool, I found that the cert was actually installed in MY profile. It was showing as part of the local machine store, but was in C:\Documents & Settings\me\Application Data\Microsoft\Crypto\RSA. It was actually supposed to reside in C:\Documents & Settings\All Users\Application Data\Microsoft\Crypto\RSA. Well, that would explain why it worked fine for me, now we just needed to find out why it was going there.
When fighting with a problem, I often find it's easiest when I've hit a roadblock to try and rethink the issue. Importing the cert via the wizard wasn't generating any errors, but it also wasn't putting it into the right place. All of the tools were giving me error messages or information that lead to dead ends. On a whim, I decided to change my first step and instead of running the wizard by double-click it, I'd trying importing the cert directly into the Certificates MMC. Now, a quick word of instruction: you can't just do Start -> Run -> certmgr.msc as this will just take you to your personal store. You have to launch MMC, add the Certificates snapin and choose Computer Account, Local Machine in order to get to the store you want.
When I did the import this way, I got a completely different error:
An internal error occurred. This can be either the user profile is not accessible or the private key that you are importing might require a cryptographic service provider that is not installed on your system.
Aha! I love different errors, especially when they lead me to the solution. So, after reading this article, I checked out the permissions on the RSA folder in All Users and sure enough they were different. Oddly enough, they seemed less restrictive than what's listed in the article, but I decided to set them as written to test. And, it worked! When I installed a cert this time, it went into All Users. If I ran the winhttpcertcfg command above, it would show that the users listed in the article were the ones who had access to the cert. In other words, all winhttpcertcfg really appears to do is set NTFS permissions on files. Ah, obscurity!
Okay, solution's now in hand, the day's halfway over, release is tonight and I have 75 servers that all need to have their configuration updated...whatever does one do? You break out the time honored skill of writing batch files, of course! I compiled all of the files I would need to make this work right into a folder. When running scripts across multiple servers, consistency is key. In this environment, I've striven for as much as I can, but these some servers are cycled on an annual basis as we retire old hardware. Consistency's an ideal, but not implemented perfectly. Compiling what I need and pushing it out to each server as part of the process really ensures it'll work.
The first batch file, setupfixes.cmd:
rd /s /q \\%1\l$\certs
I run this by issuing a for loop:
for /f %x in (serverlist.txt) do setupfixes.cmd %x
serverlist.txt contains simply a list of servers, one per line. As the loop iterates through, it simply replaces %x with the server name. This batch creates a new folder on the remote machine, copies all of the necessary files into it, runs a second batch using psexec from Sysinternals and then deletes the folder and all files since it contains our certs that we want to restrict access to. The second batch, permfixes.cmd:
cscript //h:cscript
L:\certs\xcacls.vbs "T:\Documents and Settings\All Users\Application Data\Microsoft\Crypto\RSA" /G everyone:;E1234589B /G Administrators:F /G SYSTEM:F /T /S /I Remove /SPEC B
winhttpcertcfg -g -i L:\certs\cert.pfx -c LOCAL_MACHINE\TrustedPeople -a Administrators -p password
winhttpcertcfg -g -c LOCAL_MACHINE\TrustedPeople -s cert -a "domain\client users"
winhttpcertcfg -r -c LOCAL_MACHINE\TrustedPeople -s cert -a everyone
winhttpcertcfg -g -c LOCAL_MACHINE\TrustedPeople -s cert -a Administrator
winhttpcertcfg -r -c LOCAL_MACHINE\TrustedPeople -s cert -a Administrators
I'm using xcacls.vbs because the permissions from the article are more granular than what can be set by cacls. Again, our build process specifies we set the scripting host to cscript, but that doesn't guarantee everyone's done that. Easy enough to ensure in advance, though. I set the permissions on the RSA folder to match what MS says is the default. I'm honestly not all that happy about these are they're fairly open, but I needed to get this out to production. I push these perm changes down the tree and tell it to remove inheriting permissions from higher levels. Finally, we import the cert, grant access to the users we want to have it, remove admins and everyone, and add the local admin.
Why remove admins, but add the administrator? We want to restrict access to this cert to a limited set of users. Admins like me shouldn't be able to use it as it would give them escalated privliges in the environment. But, we might need to do something with it at a later date, so we leave a back door. Since login as local admins is an audited event, we'll be notified of any potential security breach.
I've tried to include as much detail in this solution as possible because I had a hard time finding it myself. My hope is that when someone googles either "Error: Access was not successfully obtained for the private key. This can only be done by the user who installed the certificate" or "An internal error occurred. This can be either the user profile is not accessible or the private key that you are importing might require a cryptographic service provider that is not installed on your system." they'll come here and be able to fix their issues. I've put the errors in twice to make sure it gets indexed well. :)
Friday, January 1, 2010
My first iPhone Webapp
At work, I am a serious scripter. I try to use VBscripts for almost every repetitive task I do for the simple reason that I make mistakes, computers don't. If I have to edit the properties for 200+ Citrix-published applications (a task I've had to do numerous times), the chances of my breaking things is much greater than if I just get it scripted and tested once and let the script do the heavy lifting.
But, VBscripts just haven't been cutting the mustard for some of the more onerous tasks in recent months, and so I took advantage of the fact that Microsoft offers Express editions of Visual Studio 2008 as free downloads. (Recently, my company got everyone in my group MSDN subscriptions, so I was able to get VS2008 Pro to use!) I had been wanting to get more familiar with Visual Studio and app development in general as I had a couple of things I wanted to try doing. Figuring the best way to learn was to find a project I wanted to do and dive it, I thought about it for a while until I came up with...
For years my home has been controlled via a home automation system called HomeSeer. HomeSeer's a great product, put out by a company that stands firmly behind their products, with a great user community that's always ready to help. One of the pieces of HomeSeer I've played around with over the years is its scriptability. It has its own .Net-based webserver built in and will run fairly standard ASPX pages as well as VBscripts. By just adding some references into your project, you can also control it via other languages as well.
And then, of course, there's my ubiquitous iPhone. Last year I decided to give iPhone development a try and so got a cheap Mac, downloaded the SDK and gave it a shot...and realized after looking at the docs for about 10 minutes that Objective-C is not an easy transition for a VB guy to make. So, I put it on the sidelines for a while. Recently, though, I came across iUI. This is a framework put together by the founder of Facebook that allows developers to very easily create iPhone-based Webapps. Since it's agnostic to the underlying development framework, I had found my project: I would create my own iPhone interface to HomeSeer.
I see that others have done this, but I had some specific needs I wanted to accommodate. And, I wanted to learn. Creating this from scratch would give me a lot of experience with so many different areas, it was too good to pass up. And, so, I give you....
Looks pretty cool, right? It doesn't actually do anything yet, but it's got a lot of stuff going on behind it. First, it connects to HomeSeer and enumerates all of the devices and gets their status. Then, it generates this nifty page with a button that's named the opposite of the device's status. Eventually, the plan is that you'll be able to tap one of those buttons and turn lights off and on. But, since HomeSeer is very extensible and can be used to control almost anything, so too will this app be able to do a whole lot more.
Not too bad for an hour's worth of fiddling. In coming installments, I'll document what I did to get everything setup and in place to make the magic happen because a lot was either poorly documented or the documentation was spread over multiple locations.
Sunday, November 15, 2009
It's all a series of tubes, Part 1: Ping 'em all and let THEM sort it out.
Recently, I became the licensee for an exciting event called TEDxRochester. As a result of this event, I've begun networking with a lot of the great folks in my area. Prior to this, I'd used social networking sites primarily as a vehicle to share pictures of my daughter with friends and family. I had a Linked In account, but it barely got any use. Now that I've got a lot of folks to keep up with, and who are interested in keeping up on me, I'm working on changing that.
One aspect that's taken a bit of my time is aggregation. I've started following a bunch of people on Twitter and I've ramped up my connections on Linked In. Aside from this blog and the one at TEDxRochester.com, I also write reviews for iPhoneTunes.net. And, on top of that, I've still got my private connections on Facebook to keep track of. Information overload, here I come!
I use an RSS feeder everyday, so it only made sense that I should try to find a way to leverage what I'm already using to keep track of people. When I first created this blog, I added it to my Linked In profile via its RSS feed. After all, as the first entry stated, this blog was to serve as a living resume for potential future employers should I ever have need of one. (Yes, I know, I should probably update it more often then.) At the time, you could only add one blog or RSS feed, and I wanted the ability to aggregate everything I write into one, easy to use feed. I also want to keep these many different groups informed of my doings, but it has to be easy to manage. After all, I've already got a full day. Adding more "work" just to keep track of people just wouldn't fit.
Since I knew that all of these activities would result in my needing to update my statuses a bit more frequently, I decided to start by simplifying that task. For example, if I wrote a new review at iPhoneTunes.net, I'd typically post a link to Facebook & Linked In. But, now that I've actually become active on Twitter, I would need to tweet this new review as well. If I wrote a new article for the TEDxRochester blog, I'd have to post it to Linked In, Facebook, the TEDxRochester fan page on Facebook AND Twitter. I just knew there had to be an easier way to do all of this, and there is...it's called Ping.fm.
Ping.fm acts as a central dashboard to the status boxes for all of your social networking sites. At last count, they support some 60ish services like Facebook and Linked In, but also some distant, esoteric ones I've never even heard of. For me, flexibility was key, and Ping.fm provides it in the form of posting groups. These allow you to tie different social networking sites together so you can have granular control over who sees what. There are some things that I only want to share with a very limited, private group, and there are some that the whole world can see. I accomplished this by creating the following groups:
Professional: This group is where I post things that I want to be available to everyone. Anything posted to this group is visible to the world, so family vacation pictures wouldn't go here, but if I found an interesting article about some new technology I could post the link in this group. Anything posted here goes to Linked In and Twitter. It also goes to Facebook as well. After all, being a geek, I have a lot of geek friends who might not pay attention to my Linked In profile.
Private: For the moment, this one just goes to Facebook. I could do Facebook by itself, but I'm getting myself in the habit of using the group instead, just in case I decide in the future to add something else.
TEDxRochester: This is where I post news about next year's event. I also post random things that might in some way be related. For example, my wife and I were watching The History Channel's "The Universe" last night and Adam Frank, who is regularly on the show, was explaining about pulsars and quasars. I watch shows like this all the time where they bring on experts, but since Adam was our first speaker at the inaugural event, he was the first one I've ever met from one! So, I posted a quick item about it. This one's my biggest group, it posts to Facebook, the Facebook fan page, Linked In and Twitter.
I also have a del.icio.us account, so any time I post a link to any of the above groups, it's automatically added to my bookmarks. How cool is that?
It didn't take long before I fell completely in love with Ping.fm for updating my statuses (statusii?). It's very well laid out, very easy to use and free. I now spend as much time updating up to five different sites as easily as I would any one of them, if not easier. If you've got a lot of social sites to update, you owe it to yourself to check it out.
Okay, now that I've got outgoing information organized and easy to control, it was time to tackle incoming. Being only one person, though, outgoing was a lot easier...
Subscribe to:
Posts (Atom)