HUAWEI mobile connect - USB Fake driver

Two days ago I have bought mobile internet for my laptop from Swisscom (the biggest swiss telco company), and I have got a HUAWEI modem for that.

I went home happily and installed the  "Unlimited Data Manager". After the installation it prompted me to restart and I said yes. Well, the first bad sign was, that after the restart my Windows 7 machine just showed me a black screen. After waiting for a while I have forced the computer to shut down and I have started it again. This time it has started but when I plugged in my beautiful new HUAWEI modem it was not recognized correctly (in the device manager I saw HUAWEU mobile connect - 3G USB Fake) and I had no access to the internet.

As a blond girl my first attempt was to restart the machine. Well, the second symptom was that my machine did not want to shut down any more. After waiting for a while it just  crashed while shutting down (blue screen, dumping physical memory to disk....).

Then my machine did not really want to start. I took him ages. If the modem was plugged in while starting then at least it was properly recognized. But if I took it out and then put it back nothing again.

So all in all the situation looked like the following: I could connect with my HUAWEI modem to the internet only if I have freshly restarted my machine while the modem was plugged in. However my machine got sick somehow and it did not want to shut down nor did it want to turn on so a restart took half an hour at least (and this is not exaggeration at all....). 

So after two days of struggling, almost getting a nervous breakdown, crying a bit, wanting to throw my machine out the window and so on I have found the solution. I have found it here, but it is deeply buried in the comments, and I thought that it definitely deserves to be in a blog post.

So the reason why my HUAWEI modem drivers could not be installed correctly, is that I have also some VMware products installed on my machine. Apparently VMware and HUAWEI modem do not like each other at all.

The solution was to stop the following service: VMware USB Arbitration Service. I did not only stop it, but set the startup type to manual. Then I have rebooted my system once again, and this time it was willing to shut down and turn on again. After my machine was up, I plugged in the modem, and the drivers were successfully installed and I could connect to the internet. After that I was so happy that I thought I will write a whole blog post about it. :)

Efficient reporting in a web application

I know that everyone who has a webpage has Google Analytics as well, and monitors the visitors, clicks and many-many things. Google Analytics is free and cool and really great. (BTW, if you would like to become a real expert in web analitycs check out this blog.) However I have missed one thing very much: I can not track with it how users actually move through my site. I mean that I would like to see that a user arrives to my site, first she views page A, from page A she goes to page B and then to page C and then she leaves. If you want to understand your users I think it is nice to be able to see how they exactly move from page to page. Because of this, I have decided that in addition to using Analytics I will as well do a very simple reporting thing that will basically log every request with a user id and like this I will be able to check user movements. :)

My biggest concern was performance though. I did not want to make a database write for every request, as I thought it would make the read requests slower. So here is what I have done:

I have created a UserActivity table with columns like: IPAddress, Uid, UserName, CretOn, Url. If a user is logged in then I put the username in the UserName column, it is empty otherwise. If user is not logged in only Uid is filled out with a guid I generate for everyone who visits the site for the first time. Than I store the Uid in a persistant cookie, so that I can track the movement of unauthenticated users as well.

Ok so, the interesting part is how do I insert the data in this UserActivity table?

• Reporter class: Reporter class has an Activities property that is a list of UserActivity. The class has an Add method that puts an activity in the Activities list, and it has a Report method, that inserts everything into the database from the Activities list and empties the Activities.

          public void Add(UserActivity activity) {
              lock (this)
              {
                  Activities.Add(activity);
                  if (Activities.Count >= ReportingTreshhold)
                  {
                      Monitor.PulseAll(this);
                  }
              }
          }
  
          public void Report() {
              List copyOfActivities;
              while (true)
              {
                  try
                  {
                      lock (this)
                      {
                          Monitor.Wait(this);
                          copyOfActivities = Activities;
                          Activities = new List();
                      }
                      // Report the Activity to the database
                      ReportingEntities reportingdb = new ReportingEntities();
                      foreach (var item in copyOfActivities)
               {
                          reportingdb.UserActivities.AddObject(item);
                      }
                      reportingdb.SaveChanges();
                  }
                  catch (Exception ex)
                  {
                      ExceptionHelper.RaiseErrorSignal(ex);
                  }
              }
          }
  

• Start a background thread in Global.asax: For the reporting thing to run I start a background thread when the application starts in the Global.asax.

     Thread reporterThread = new Thread(new ThreadStart(Reporter.Report));
     reporterThread.Start();
  

• As I have an ASP.NET MVC application I have created a ReprotingFilter that runs on every request and adds a UserActivity to the Activities list.

          public void OnActionExecuted(ActionExecutedContext filterContext)
          {
              UserActivity activity = new UserActivity();
              HttpRequest currentRequest = HttpContext.Current.Request;
              // We filter out search engines
              if(!currentRequest.Browser.Crawler){
  
                  // Create the UserActivity Object here...
    
                  Application.Reporter.Add(activity);
              }
  
          }
  

Actually I am absolutely satisfied so far with this reporting thing. My site is still quick, user requests are now logged. There was only one big mistake I have made for the fist time: Do not forget to filter out search bot requests, otherwise you will see logged tons of requests from search engines!

How Google, Facebook and forms authentication can work together?

According to my experience a real word authentication thing is till not very easy if you want to support 3rd party logins (like Google and Facebook) and traditional forms authentication and you want them to work together. So what do I mean by working together and real world?

• In the real word you will want to collect some extra information about users when they register or login via 3rd party for the first time. For example in my case I make a beauty site so I collect user's skin type and biggest skin concern.
• It is nice if different login options can work together: users can connect their accounts to the one profile they have on your site and after connecting the accounts they can log in with whatever method they want. This is especially needed if you add 3rd party login support after the site is launched and you have already users registered in the traditional way.

So how do you implement something like this with ASP.NET MVC?

• You need for this the right database structure: To support forms authentication I just use what comes out of the box, the asp.net membership system with sql membership provider. To collect extra information about the users I have created my own user table that contains skin type, skin concern, nick name and so on. To support the one profile can have many logins I have created a connection table that has two columns: nick name that points to my own user table and username that points to the user in the membership system.
• In case of 3rd party login I also create a membership entry for that login. For username I use the unique identifier of the user I get back from Google or Facebook. Like this I know if user is logging in for the first time or not: if the unique identifier has already a membership entry, it is not the first time otherwise it is.
• If it is a first time login I tell the user to create a profile on my page or ask if she has already a profile. If the user creates a profile I make an entry in UserProfile and also in the UserNickConnection table. If user has already a profile, I ask her to authenticate as before and then I just have to make a new entry in the UserNickConnection table. After the accounts are connected user can log with whatever method she wants.

Ok, so this was the high level overview of the thing. To get started more concreatly the following things are needed:

• To support Google login I user the DotNetOpenAuth library. Also to figure out how to do it the source code of the nerddinner site is great.
• To support Facebook connect I use the Facebook C# SDK. It also has an example and based on that it is easy to make it work.

Nice Urls - Slugs

By default ASP.NET MVC comes with urls like the following: /products/details/256. I find this already some kind of improvement over the normal web forms urls with aspx filename in the end but having numbers in the url is not really fashionable and SEO compatible. So while refactoring my app I thought that I will also make some fashionable and beautiful urls using so called slug. A slug is url compatible (no special characters, no spaces) string that is generated usually from the title of the item. So with slugs you can have url like /products/details/back-pack. It is not at all so hard to generate slugs. So you need an extra colums in the database table and on creation you generate the slug than you just need a GetBySlug instead of GetById and it is all working. If you are like me and have started out with ids and then change for slugs and want to keep also the "old" urls working the best is to have a GetByIdorSlug that tries to parse the third part of the url, and if it is an int it gets by id otherwise by slug, and that way old and new urls will work. :) Ok enough talking. I get to the point. Here is the code how to generate slug:

 public static string RemoveDiacritics(string stIn)
        {
            string stFormD = stIn.Normalize(NormalizationForm.FormD);
            StringBuilder sb = new StringBuilder();

            for (int ich = 0; ich < stFormD.Length; ich++)
            {
                UnicodeCategory uc = CharUnicodeInfo.GetUnicodeCategory(stFormD[ich]);
                if (uc != UnicodeCategory.NonSpacingMark)
                {
                    sb.Append(stFormD[ich]);
                }
            }

            return (sb.ToString().Normalize(NormalizationForm.FormC));
        }

        public static string CreateSlug(string stIn) { 

            string slug = stIn.ToLower();
            slug = RemoveDiacritics(slug);

             // Replace unwanted characters with space
             slug = Regex.Replace(slug, @"[^a-z0-9\s-]", " ");
             // Replace multple white spaces with single space
             slug = Regex.Replace(slug, @"\s+", " ").Trim();
             // Replace white space with -
             slug = slug.Replace(" ", "-");
 
             return slug;
        }

Things I wish I knew about how to create an ASP.NET MVC app....

I have a website I am doing as a hobby, and it is an ASP.NET MVC web application. I have started to write the code in March and I have launched the site in July so it is not very small not very large either, it is a normal size web app. :) I am full of plans though what new features to implement and even since July I have done already three new versions so the app is getting bigger and bigger. It has now reached a point that I have started to feel that the application is not well written enough to be maintainable in the long run so I have decided to refactor the whole backend. Here is a list of things I wish I knew when I started the app:

• In many sample databases and apps (at least in the ones I have seen before starting to do my app) the naming convention for the db table id is TableNameID (ProductID, OrderID...). This naming convention is a piece of crap. It is much better to call all ids just simply Id, and then you can make an interface IHasId and have nice methods that handle objects that implement IHasId. And I tell you to rename table ids is really painful though it is not impossible....
• LINQ to SQL was basically replaced by Entity Framework, and EF is developed by Microsoft while LINQ to SQL is just in maintenance mode... So if you can switch. I have switched....
• Dependency Injection is nice and it is nice to use an IoC container. If you want to understand what is this whole stuff I suggest this great presentation from Brand Wilson. I am using now Ninject.
• A Service layer is great to have. When I have started to do the app I have thought that only large apps need service layer and anyway I have a simple web app, no business logic. But it turns out there is always some business logic and service layer is good also in not so large apps...
• There is a great starter ASP.NET MVC app made by Rob Conery on codeplex. It is nice to download it and study it a bit and lean form it. The generic CRUD repository that is usable for all enities is especially great. Before I had All, Single, Add and stuff like that for all my entities separately, and this starter app taught me how to have just one generic. :) I love if I can delete code from my app and make my app cleaner with less lines of code. :)
• Never trust yourself or EF or LINQ to SQL and always check what gets actually executed on the database. I think that these abstractions and writing queries in C# and using navigation properties is great but you can find yourself very very easily doing very ineffective things on the database level (like select N+1, really ineffective queries with tons of joins....). Always monitor what is going on in the database and entity framework profiler is a great tool for that.

GIT branching and merging

Further useful git things about branching and merging: To list your existing branches:


$git branch


To check out visually what your branches are doing:


$gitk


To create a branch:


$git branch newbranchname


To change to a certain branch:


$git checkout branchname


To merge the changes made in the branch "branchname" into the current branch:


$git merge branchname


After merging the branch is not deleted automatically. To delete it:


$git branch -d branchname


To undo a merge:


$ git reset --hard HEAD


Merge branch obsolete into the current branch, using ours merge strategy:


$ git merge -s ours obsolete

Take out of source control some directories:

$ git rm -r --cached Somefolder/bin

An untraditional case with jquery autocomplete plugin

In my hobby project I have been using the jquery autocomplete pugin so far in several situations and I love it. It has so far fulfilled all my requirements, about every wish I had it turned out that there is an option to do that (e.g. multiple results, formatting, caching....). I know this plugin is not any more maintained but the new official jquery ui autocomplete is just so dumb compared to this one so I have decided to wait with the migration and continue to use this one (hopefully with time the jquery ui autocomplete will also get smarter....). So this hapiness was all true until today when I wanted to implement the following:

As the screenshot shows I wanted to have a small form with three inputs. In the third input I wanted to have an autocomplete that gives back the suggestions not only based on the prefix but based on the value of the first two inputs. So far this was no problem for the autocomplete: there is an option called extraParams that can be used for that. However I wanted to give back the suggestions as soon as the user clicks into the third input because I can restrict the possible values quite a lot already based on the selection in the first two fields. So triggering autocomplete on focus was my wish and this time my favorite plugin has let me down. There is a search() function that looked promising, so my first try was the following:

$("input#creme").focus(function(){
   $(this).search();
});

And indeed it does trigger a get request from the server however the ui was nowhere. :( My second try was something like this:

$("input#creme").focus(function(){
   $(this).trigger("keydown.autocomplete");
});

This one has not event sent a request to the server. It has done just nothing, and I do not know why. So after being sad the only hacky solution that has helped was that I have actually opened up the source code of the plugin and hacked into it so that with search I can trigger the autocomplete. I had to change just one line: in the search handler function I have replaced

request(value, findValueCallback, findValueCallback);

with

request(value, receiveData, hideResultsNow);

and this has actually helped. As I am not using search anywhere else, it is just ok for me now. However you will have to find out something smarter if you use search() according to its intended purpose. To get a perfect solution I just had to do one more thing: if the user changes the value in either of the first two inputs then the autocomplete cache has to be flushed.

$("input#brand").focus(function() {
   $("input#creme").flushCache();
});

I am flushing the cache already on focus. I am not good with javascript, but if you are here I guess you have already found that out. :) (I am planning to learn it though! :))