Improving The Web

Wow, I can’t believe it’s been this long since I last posted on this blog. I had so much plans for this blog, yet most (if not all) did not materialize.

The reason for this is that I was struck by chronic headaches, neck pain, eye pressure and tinnitus.

As you can imagine, these conditions are not a good match if you’re working on a computer all day long. That’s why I decided it was time for a break. I wasn’t able to work in a productive manner anyway.

Now that I’m feeling better (and after a lot of doctor visits), I’m slowly returning to my normal work routine. I still haven’t got rid of the tinnitus, but at least the headaches are gone.

First matter to attend to is to update some of my plugins so that they are compatible with wordpress 2.8 / 2.9. I got called out on this a few days ago on WeblogToolsCollection and decided this was a good reason to get back to working on this site.

So, with that said, the plugin Improved Plugin Installation has been updated and should now work in all wordpress versions.  Apologies to all those who have waited so long for an update.

About two weeks ago I created a new website: ConvoTrack.com.

This website offers a bookmarklet that, when installed, allows you to see the entire conversation surrounding a particular webpage at the press of a button. Comments from Twitter, FriendFeed, Digg, Reddit, HackerNews and any blog mentioning the article are loaded in a sidebar. This way, you can see comments that were never on the original site.

Additionally, you can send a link with the sidebar preloaded to people who don’t have the bookmarklet installed. Simply prepend “http://convotrack.com/” to any URL to do so.

Site integration

If you’d like to add this functionality to your own website, you can use the code found on ConvoTrack.com. This will add a button on the left side of your website that behaves exactly like the bookmarklet.

If you’re a wordpress user however, you may be interested in the more advanced Backtype Connect wordpress plugin. This plugin allows for a seamless integration of the outside comments in your own comment system.

How was it built?

BackType.com is the service that does all the heavy lifting. Using their newly released BackType connect API the bookmarklet queries for all comments pertaining to a particular URL and then outputs them in the sidebar. Very easy.

What’s coming?

A firefox extension should be released soon, this will allow you to have the sidebar visible at all times and will update the comments as you browse the web.

Suggestions?

If you have any suggestions for improvements, do tell.

Well, it’s time to unveil what kept me away from this blog the past few weeks. The twitter mashup I was talking about is now live and is called TweepleRank.

At the most basic level, it tracks user recommendations made on Twitter. To accomplish that, it tracks a couple of hashtags, namely #followfriday, #mrtweet and #tweepletuesday. If you don’t know these, that doesn’t matter, but basically each of these allows you to recommend other tweeple.

The website is basically a tool to help you find new tweeple to follow. First, you have the top recommended tweeple lists which showcase the top tweeple in specific time periods. For example, here are the results for the first #followfriday. Or how about the top recommended people for the entire year?

A general top list is fine, but I want to be able to search and find recommendations that are personal to me. Luckily, the website allows you to do that as well.

You can search either the tweets themselves, the user bios, or user tags. What are user tags? Well, user tags are any and all hashtags that have been added to the tweet in addition to #followfriday. For example:

#followfriday Check out @improvingtheweb #php #entrepeneur, @jdevalk #wordpress.

Now, I (@improvingtheweb) am associated with the tags “php” and “entrepeneur”. @jdevalk is now associated with “wordpress”.

The great thing is that you can use this website to see the top recommended people for whichever tag you want. The results will be sorted by either the number of recommendations or the number of followers.

For an example, see results for the tag #travel.

If you want to see a full list of tags, simply go to this page.

Finally, one last feature: personalized results.

This allows you to see the top recommendations in your network: made by the people you follow and no one else.

This is interesting because those recommendations are probably much more in tune with your personal interests and can lead to some interesting new tweeple to follow.

That’s about it, I’ve got a few other features that will launch next #followfriday, so stay tuned! If you have any suggestions, let me know in the comments!

Well, I failed one of my new years resolutions: Blog at least once a week.

That goal wasn’t even ambitious and yet I failed it anyway.. I had a whole of 2 posts last month. In all fairness, I had been sick for about a week and a half in the beginning of the month.

Anyway, I’m pretty excited about some projects I’m currently working on. I have put some wordpress plugin ideas on the back burner and instead am focusing on two twitter mashups. Those should be released this month.

Those looking for wordpress plugins shouldn’t worry, I have a few available that just need a few tweaks and are ready for release.

As a developer, I continuously scan the internet looking for new ideas. A big part of that is spent simply reading forums, discussions, etc, to see what problems people have online and how to solve them.

However, I’m also subscribed to a big list of blogs and directories that review the newest web applications. Just seeing these can be inspirational and can suddenly give me a new idea.

To that end, I have compiled a list of what I think are the best sites to subscribe to:

Blogs

TechCrunch
TechCrunch needs no introduction. The go-to source for the latest and greatest news.

ReadWriteWeb
Similar to TechCrunch, with more emphasis on reviews and analysis.

Mashable
A leader in the web 2.0 and social networking news space.

Killer Startups
Reviews 15+ web startups per day. List pros and cons for every idea.

A Startup A day
Excellent resource. Lists ideas that haven’t yet been converted into web apps.

Idea Shower
Another excellent resource. The owner shares ideas online and you can follow the development progress.

More:
WebWare
Profy
Go 2 Web 2.0
Make Use Of

Directories

eHub
One of the first “web 2.0″ directories.

Trade Vibes
Provides lots of information on startups. Can be used to quickly find competing or related services as well.

Listio
Similar to TradeVibes.

More:
The Museum of Modern Betas
Feed My App

Community

YCombinator News
Y-combinator is a VC, this is their Digg like site. Pay special attention to web app review request threads, these can be very insightful.

Clusterify
This website just started, but it has the potential to do great things. It’s basically a way to meet other developers and work on small projects together.

Sites For Sale

Sitepoint Marketplace
Looking at websites for sale can give you great insights into what is monetizable and what isn’t.

Trends

Delicious Hotlist
The delicious hotlist gives you a quick glance of what is popular on the web right now.

Design

Design Meltdown
This design gallery lists items by color, design element, etc.. Excellent categorization.

More:
CSS Beauty
Web Creme

Niche

Inside Facebook
Facebook app reviews and more.

Refreshing Apps
Focuses on Adobe Air application reviews.

Weblog Tools Collection
Look for the weekly post listing the newest wordpress plugins and themes.

AppVee
Text and video reviews for iphone applications.

If you know of any other great resources, do let me know via the comments.

MyBlogLog is a service that dates back quite some time. I’m sure you’re all aware of the “Recent visitors” widget displaying the latest MyBlogLog members to have visited your site.

It’s a great concept, but there are a few things I don’t like about it:

- You only see the last X people.
- There’s not a lot of profile information disclosed.

To remedy this, I have created a new Wordpress plugin: Track MyBlogLog.

It does two things:

Track users

Similar to the widget, it will track all MyBlogLog members that visit your site. However, it will save the results in the database so you can see more results and can go back in time. You’ll also be able to see just how many times these MyBlogLog readers come to your site.

Additionally, the plugin will try and find more information about the user, such as their website URL, twitter URL, location, etc..

Identify users

The second part of the plugin simply makes available the information found before (website, name, etc..) and allows you to create a personalized message for the user.

You can see an example in the screenshot below:

Privacy Issues

I would never release this plugin if I thought there were privacy issues. I don’t believe there are. MyBlogLog users are very clear on the fact that they are being tracked and shown in the “Recent visitors” widget. This is just an extension of that so there shouldn’t be a problem there. MyBlogLog also provides controls to opt out of the tracking service.

Furthermore, most MyBlogLog users are also webmasters, and to me this seems like a win-win situation for both parties. The visitor will get more traffic back to his site, and the site administrator will learn more about his readers.

TwitterRemote

Yesterday a new widget was released allowing you to see which twitter users are visiting your site. Very similar to MyBlogLog in that regard. The only downside is that this is a third party service and you need to log into it, which means this widget can’t ever track the whole twitterverse.

Still, I’m thinking about making a similar plugin for this widget, if there is demand for it.

Download

For download instructions, go to the dedicated plugin page.

BackType, as I’ve already explained in a previous post, is a comment aggregator. If you sign up, the service will track all the comments you make across the web.

BackType is infact a search engine. This means you don’t have to install any browser extensions or anything of the like to track your comments. The entire process is transparent and automated.

Using BackType’s API, I’ve created a wordpress plugin that lets you import your comments into your blog. On set intervals, (hourly, daily or weekly) the plugin will download your newest comments and add them to your database.

This is a great way to show your visitors what you’re doing on the web and find interesting. As an example, you can see my comments here.

In addition to the paginated listing of comments, a widget is also available for your sidebar. Lastly, you can set it up so that a digest of your latest comments is automatically posted either daily or weekly.

For more information and download instructions, go to the dedicated page: My Comments Elsewhere.

There’s been numerous times I wanted to continue to follow the comments made on specific blog posts.

If I was lucky, the blog owner had the Subscribe to comments Wordpress plugin installed. The problem is that only a small percentage of blogs actually do.

The solution?

Let BackType handle everything. BackType works across all blogs, be it Wordpress, MovableType, etc.. You simply install a bookmarklet in your browser and click on it whenever you’re on a page you want to follow.

You can then subscribe to a RSS feed that will update either in real time, once a day or once a week with the newest comments from blog posts you’re following.

Having the results returned as a RSS feed makes everything much easier to manage. The old Subscribe to comments plugin sends you a bunch of emails instead, cluttering your inbox. Secondly, everything is centralized at BackType. This ensures you can easily add or remove subscriptions in 1 place.

But that’s not the only service BackType provides

In fact, it’s core service is to track all comments across the web. These are then indexed and made searchable on their website. A wealth of knowledge is contained within.

The best part for me is being able to follow other people. For example, here is my profile. If you want to, you can follow me and from that point onwards you will see every comment I make. Very powerful. Here are some popular people.

There’s one other service; Alerts. Think Google Alerts, but for comments. Whenever the keywords you specify are in a comment, you’ll receive a notification.

Now, BackType is still a pretty young service, and if there’s one fault with BackType, it’s that their index is not complete. However, improvements are made daily. If you find your site is not yet added to the database, you can do so here.

Do you read TechCrunch? If so, you’ve probably noticed how they integrate their other internet properties into the site in quite an ingenious way.

Several of the articles listed don’t link to subpages on techcrunch.com, but instead to some of their other sites; CrunchGear, MobileCrunch, etc..

Now, I’m not sure if this already exists on WPMU or not, but I wanted something similar for a normal, standalone wordpress installation.

That’s why I have created yet another plugin; External Permalinks.

You can use it in a variety of ways:

- As detailed above, to cross promote posts across your blog network.
- To link directly to a guest post author’s blog
- Etc, etc..

It’s excellent for doing traffic exchanges with other blogs; each blog simply posts an excerpt for an article on the other blog, and adds the external permalink.

This sends you some excellent quality traffic and no doubt alot of new subscribers.

The permalink is also changed in the RSS feed, so when the reader clicks on it to make a comment, they are automatically redirected to the external URL.

For download instructions, go to the dedicated page: External Permalinks.

Now, couldn’t you just include the external URL in the post text? Yes, but you won’t get the seamless integration this plugin provides.

With this plugin, you’ll get more clicks to the external site, especially in RSS readers as users most often click on the permalink URL or comments URL, both of which are redirected to the external URL. (Comments page is optional though)

You may not know, but I manage another server.

Earlier this week that server got really slow for some reason, so I rebooted it. I waited and waited but it didn’t start up again.. After some investigation, it appeared the hard disk had crashed.

I knew I had backups, so I thought I was safe. I asked server support to put the backup online. Unfortunately for me, it appears the last backup had not succeeded! Yes, disaster struck twice, and at what a time…

I asked them to try and recover the data from the failed HD. Luckily they were able to get the database, which is the most important thing on the server. Without the database, I would have lost all member info, forum data, etc etc.. The site would have been worthless.

Unfortunately, they couldn’t recover anything from the /home partition. This means a ton of user uploaded images were all lost. Well, not all of them, the backup HD did have an older backup available so I was able to save about 70% of the images. Still, the loss is enormous.

I’m now thinking about sending the HD to some sort of professional data recovery service. Something like DriveSavers. Pricing for this kind of thing is enormous though, ranging from $1-$2,000. What a lucrative business that is.

Could I have prevented this from happening?

Yes, I probably could have. I should have read my logwatch reports from cpanel. In the last logwatch report before the crash, it did indeed state there were some HD issues, but I hadn’t read it.

Good news is, those kinds of errors are almost always listed at the top of the logwatch email. Here’s what it said in my case:

WARNING: Kernel Errors Present
I/O error: dev 08:08, sect...: 35Time(s)
Additional sense indicates Unrecovered read error - auto reallocat...: 35Time(s)
Current sd08:08: sense key Medium Error...: 35Time(s)
EXT3-fs error (device sd(8,8))...: 53Time(s)
ata3: error=0x40 { Uncorrect...: 1Time(s)
ata3: status=0x51 { DriveReady SeekComplete Error }...: 194Time(s)


But why did the backup fail? After some investigation, it appears cpanel tried to gzip all the files, and there simply wasn’t enough room left on the backup HD.

As the site grew, so grew the number of images. The backup HD had gotten too small. I hadn’t a clue anything was wrong because each day I happily received the email titled “Backup complete” from cpanel. I investigated the last backup log email from before the crash, and yes, on line 1022 this was stated:

gzip: stdout: No space left on device

1 line out of 1500 to inform me the backup had failed. GREAT.

Are you safe?

Do you read your logwatch reports? Your backup logs? Logwatch reports aren’t only useful to see if something is wrong with the HD, they also list possible hacking attacks, so they shouldn’t be ignored. Ofcourse, I’ll have to follow my own advice from now on.