Monday, July 12, 2010
Mainly for me to remember this but if you’re using the excellent Versioning bundle for RavenDB then you’ll find you quickly need to work out a way to get the latest version of a document if querying on any field other than ID (I *believe* ID actually only gets the latest one). The little index definitions below do just that (the secome one is just adding another field to the map so I can query by that…useful example)…
documentStore.DatabaseCommands.PutIndex("CurrentVersion",
new IndexDefinition
{
Map = @"from doc in docs
where
doc[""@metadata""][
""Raven-Document-Revision-Status""] !=
null &&
doc[""@metadata""][
""Raven-Document-Revision-Status""] ==
""Current""
select new {doc};"
},true);
documentStore.DatabaseCommands.PutIndex("CurrentImageVersionBySlug",
new IndexDefinition
{
Map = @"from doc in docs
where
doc[""@metadata""][""Raven-Entity-Name""] ==""Images""
&& doc[""Slug""]!=null &&
doc[""@metadata""][
""Raven-Document-Revision-Status""] !=
null &&
doc[""@metadata""][
""Raven-Document-Revision-Status""] ==
""Current""
select new {Slug=doc[""Slug""]};"
, } , true);
Friday, July 09, 2010
So, I’m not dead! Have been busy moving house, getting two new kittens (Tycho and Cozmo) and bascially settling into my new existence.
I HAVE also been getting back down to some serious coding. LOVING RavenDB (I seriously think it is THE .NET database of choice now), getting deep an dirty with some awesome HTTP optimization techniques…which I really do have to blog about soon!
My first pointers to HTTP optimization greateness are:
Chirpy: a brilliant little VS add-in (check the Codeplex page for the VS 2010 version) , compresses and combines your CSS and JS files as well as lets you use .less syntax easily inside VS (kinda a programmatic extension to CSS allowing for cleaner, less repetitive style definitions).
LabJS: a bit more complex but really stunning little bit of JS which lets you seriously optimize your JS loading…JS becomes non-blocking and all loads asynchronously (see http://labjs.com/description.php for more details)! Here’s how I use this in one of the cozwecan sites (note, I also use LabJS to do a ‘load from CDN with fallback’ behavior which I have yet to see fully documented) …
var scriptModifier = "";
$LAB
.setGlobalDefaults({ AllowDuplicates: false });
$LAB
.script({ src: "http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js",
alt: "/Scripts/jquery.1.4.2.min.js",
test: "jQuery"
}).wait()
.script({ src: "http://ajax.microsoft.com/ajax/jquery.validate/1.7/jquery.validate.min.js",
alt: "/Scripts/jquery.validate.min.js",
test: "jQuery.validate"
})
.script("/Scripts/jquery.ba-bbq.min.js")
.script({ src: "http://cdn.jquerytools.org/1.1.2/tiny/jquery.tools.min.js",
alt: "/Scripts/jquery.tools.min.js",
test: "jQuery.tools"
})
.script("/Scripts/jquery.easing.1.3.min.js")
.script("/Scripts/jquery.jnotifica.min.js")
.script("/Scripts/Custom/MainScripts" + scriptModifier + ".js")
.wait(function () { SetupInitialLoad(); });
Anyway will get a blog post together eventually covering all that I’ve been up to…until then…follow me on Twitter @scottgal..late peeps!
Saturday, March 20, 2010
Ok, not a catchy title but it really is what I’ve been up to for the past 3 months. As I wrote before, I’m currently working with the awesome @RobertTheGrey on a project called cozwecan, which is a bunch of sites set up with the aim of being an amazing independent ecommerce platform for artists (photographers to start with, in the future, who knows…). Anyway, I’ve been remiss in blogging about the amazingly cool stuff I’ve been doing (IMHO :)), using WPF with Reactive Extensions, MongoDB, FluidKit and of course the awesome WPF / SL MVVM framework, Caliburn (the embedded bit aboveis the creator of Caliburn, @EisenbergEffect’s presentation on MVVM from MIX). So, just licking my wounds and getting my brain back in one piece after last year’s craziness (minor drug problem, clinical depression, car crashes, international moves etc…) and will be back to full communicative me soon enough :)
Monday, February 15, 2010
In the little app I’m working on I’m using MongoDB…which is great, however MongoDB only supports storing Document objects. Which are kinda like Dictonaries (Document[“blah”] = “bar”;). I want to use AutoMapper to map these loosely types Key-Value Pairs to my objects and I’m basing it on some code I found drifting around:
This works:
public static IMappingExpression<Document, TDestination> ConvertFromDocument<TDestination>(this IMappingExpression<Document, TDestination> exp, Func<string, string> propertyNameMapper)
{
foreach (PropertyInfo pi in typeof(TDestination).GetProperties(BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic))
{
if (!pi.CanWrite ||
pi.GetCustomAttributes(typeof(TDestination), false).Length == 0)
{
continue;
}
string propertyName = pi.Name;
propertyName = propertyNameMapper(propertyName);
exp.ForMember(propertyName, cfg => cfg.MapFrom(r=> r[propertyName]));
}
return exp;
}
This maps my document to an object…cool…however, I’m having real problems figuring out the reverse (my lambda / knowledge of AutoMapper are letting me down):
This is NO way works…
public static IMappingExpression<TSource, Document> ConvertToDocument<TSource>(this IMappingExpression<TSource, Document> exp, Func<string, string> propertyNameMapper)
{
if (typeof(TSource) is IBaseObject)
{
foreach(PropertyInfo pi in typeof(TSource).GetProperties(BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic))
{
string propertyName =pi.Name;
propertyName = propertyNameMapper(propertyName);
exp.ForMember(r=>r[propertyName], cfg=>cfg.MapFrom(p=>pi.GetValue(p));
}
}
else
{
throw new Exception("What type are you trying to give me???");
}
return exp;
}
Help my Jimmy Bogard, you’re my only hope :)
Saturday, February 13, 2010
I wrote this for use in a little project but as I’m no longer using it, I though I’d stick it on here for anyone who wants it. Essentially, this allows you to take a byte Array (in my case, in the File property of my ThumbnailViewModel) and get it back as a BitmapImage for use in DataBinding…
It also has the property of accepting a parameter which lets you specify the size of thumbnail to use (ThumbSize). Here, I actually have multiple ThumbNails (in the List<ThumbNailViewModel>) and select one using the parameter.
Given this resource string and XAML,
<UserControl.Resources>
<local:BytesToBitmapConverter x:Key="bytesToBitmapConverter"/>
</UserControl.Resources>
<Image Name="IconImage" Source="{Binding Path=ThumbNails, Converter={StaticResource bytesToBitmapConverter}, ConverterParameter=Small}"/>
public class BytesToBitmapConverter : IValueConverter
{
public object Convert(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture)
{
var thumbs = value as List<ThumbnailViewModel>;
var param = parameter as String;
var thisThumb = thumbs.Find(x => x.ThumbnailSize == (ThumbSize)Enum.Parse(typeof(ThumbSize),param));
BitmapImage bi = new BitmapImage();
bi.BeginInit();
bi.CacheOption = BitmapCacheOption.OnLoad;
bi.StreamSource = new MemoryStream(thisThumb.File);
bi.EndInit();
return bi;
}
public object ConvertBack(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture)
{
throw new Exception("The method or operation is not implemented.");
}
}
Wednesday, January 20, 2010
Well, OK I’m over dramatizing it a tad :) As you may have noticed from my last post I’ve started working of some stuff which is outside my comfort zone. I recently joined a company called cozwecan,working for a chap named @RobertTheGrey, the first project we’re working on is a photo sales site…you know kind of like Getty Images / Smugmug / Flickr except for direct photographer-customer sales. As the only full-time developer (so far!) I’ve been tasked with building the vast majority of the applications (be they web, desktop, etc…) which we need for the business to work.
Obviously one of the most critical bits for an e-commerce site is stuff to sell! In our case this is a stock of really high quality photographs taken by professional photographers around the world. First question; how do we get these images on the site, and how do we let the photographers add information to their photographs to let them be found by folks who want to buy them? Well, that’s the first problem I’ve been asked to solve!
When working on the requirements, it quickly became clear that this had to be a desktop app, it needed to allow operations on multi-megabyte photographs, needed to allow ‘tagging’ offline. Especially during initial loading, there’s 10-100s of thousands of images getting worked on…forcing this to be online adds a nasty lag and makes the task a pain in the butt to complete. Above all, it needed to be reliable!
Now, in my dev head this led to a number of decisions about the features the app would need to support:
1. Intuitive, responsive UI; the users are not necessarily computer-savvy, need to use obvious UI metaphors and make it fast enough to be pleasant to use.
2. Needs to do some offline image processing; we’re planning on hosting this on the cloud….CPU time costs cash :)
3. Should work offline
4. Needs to be built around reliable upload of multi-meg files (across potentially tiny connection bandwidths).
So, these are the basic needs…technically this leads me to some technology choices:
1. Multi-threading / concurrency (as with any real desktop app) is key…as is a decent Desktop technology…
2. See above, we need to ‘watch’ for new files arriving, make thumbnails transparently etc…
3. So, it needs some sort of persistence…along with other non-core app requirements (e.g. tagging) we’ll likely need a little, lightweight DB.
4. Needs ‘block’ based uploads with retry / verification of uploaded items…
Luckily I’ve been a dev for quite a while and had at least some idea where to start…the biggest challenge would be learning!
Firstly, the obvious desktop platform de jour is WPF…this is in essence the current replacement for WinForms, it’s VERY customizable but has a somewhat infamously steep learning curve. I’ve spent a bit of cash on books on WPF, followed all the PDC presentations, podcasts etc…on the subject matter. After 3 weeks I’m finally getting comfortable with it…
Second, I needed a DB…needed to be lightweight, needed to support LINQ (what, I like LINQ :)), needed to support some sort of ORM technology.
The most obvious choice was SQL Server Compact Edition, a kind of cut down SQL server which is great for desktop apps…however…I could not get it working! Turns out it’s really sensitive to the versions of various installed components. This was a red light for me…I needed the DB to be as stand-alone as possible! That left one obvious choice:
SQLite, from the website:
‘SQLite is a software library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine. SQLite is the most widely deployed SQL database engine in the world. The source code for SQLite is in the public domain.’
Perfect! For .NET support, I used the SQLite ADO.NET provider (there is a C# only port of SQLite, but it’s slower and not needed for my scenario. Deployment is literally including a reference and creating a file…now it does have one pretty severe limitation which I’ll cover later :)
My ORM choices were nHibernate or EF…there being support for SQLite in both of these platforms.
Now, I have used nHibernate for a recent project (disclaimer, I left fairly early in the project :)) and one of the things which annoyed me was the large number of dependencies, and the seeming fragility across versions of these dependencies. For a low impact desktop app, this was just too large a footprint and too big a risk. So I had to count it out.
This left me with one choice…Entity Framework. Now, this wasn’t an easy choice…I have used EF in the past (tinkered with it) and found it a painful experience…(others have too which led to this). Now, I’m not getting into the politics behind MS choice to push EF (and yes, it was a political choice), but V1 is fairly buggy, and for me at least fairly unnatural. This led to my second learning ‘opportunity’…Master Entity Framework! As I write this, it’s literally 2 hours since I worked out my last bit of EF pain…which led to this model:
Looks simple…well, yeah…but it’s taken me AGES to get to this…I know SQL really well, C# REALLY well, but the mix of obscure concepts, poor documentation and obfuscated error messages makes working / learning EF way too hard! EF 4 (in VS 2010 / .NET 4) improves this…and it can’t come soon enough!
So, that’s worked out…I hope :)
So, I mentioned previously that SQLite has a bit of an issue…it doesn’t really like concurrency. Well, that’s an overstatement…it is a multi-read, single write system. Once you realize it’s an issue it’s fairly easy to solve. I use this pattern:
public static void AddRange(IEnumerable<UploadFile> files)
{
using (PixEntities ent = new PixEntities())
{
using (new ReadLock(entityLock))
{
using (new WriteLock(entityLock))
{
foreach (var file in files)
{
ent.AddToUploadFiles(file);
ent.SaveChanges();
}
}
}
}
}
As you seen, this has two main elements…I use a ReadLock (where I would normally read *from* the DB, in this case umm, I don’t :)) then I use a WriteLock to wrap any operation where I write to the DB. The observant will also notice what looks like a bug…I do ent.SaveChages() for each iteration of the loop…this is another SQLite quirk, it doesn’t support batch updates…and gives a weird error about file locks if you try :)
The little locking class I use is from here, but I’ve put it below for your use…anyway, this is part one of several parts of my experiences in working on cozwecan…stay tuned!
public static class Locks
{
public static void GetReadLock(ReaderWriterLockSlim locks)
{
bool lockAcquired = false;
while (!lockAcquired)
lockAcquired = locks.TryEnterUpgradeableReadLock(1);
}
public static void GetReadOnlyLock(ReaderWriterLockSlim locks)
{
bool lockAcquired = false;
while (!lockAcquired)
lockAcquired = locks.TryEnterReadLock(1);
}
public static void GetWriteLock(ReaderWriterLockSlim locks)
{
bool lockAcquired = false;
while (!lockAcquired)
lockAcquired = locks.TryEnterWriteLock(1);
}
public static void ReleaseReadOnlyLock(ReaderWriterLockSlim locks)
{
if (locks.IsReadLockHeld)
locks.ExitReadLock();
}
public static void ReleaseReadLock(ReaderWriterLockSlim locks)
{
if (locks.IsUpgradeableReadLockHeld)
locks.ExitUpgradeableReadLock();
}
public static void ReleaseWriteLock(ReaderWriterLockSlim locks)
{
if (locks.IsWriteLockHeld)
locks.ExitWriteLock();
}
public static void ReleaseLock(ReaderWriterLockSlim locks)
{
ReleaseWriteLock(locks);
ReleaseReadLock(locks);
ReleaseReadOnlyLock(locks);
}
public static ReaderWriterLockSlim GetLockInstance()
{
return GetLockInstance(LockRecursionPolicy.SupportsRecursion);
}
public static ReaderWriterLockSlim GetLockInstance(LockRecursionPolicy recursionPolicy)
{
return new ReaderWriterLockSlim(recursionPolicy);
}
}
public abstract class BaseLock : IDisposable
{
protected ReaderWriterLockSlim _Locks;
public BaseLock(ReaderWriterLockSlim locks)
{
_Locks = locks;
}
public abstract void Dispose();
}
public class ReadLock : BaseLock
{
public ReadLock(ReaderWriterLockSlim locks)
: base(locks)
{
Locks.GetReadLock(this._Locks);
}
public override void Dispose()
{
Locks.ReleaseReadLock(this._Locks);
}
}
public class ReadOnlyLock : BaseLock
{
public ReadOnlyLock(ReaderWriterLockSlim locks)
: base(locks)
{
Locks.GetReadOnlyLock(this._Locks);
}
public override void Dispose()
{
Locks.ReleaseReadOnlyLock(this._Locks);
}
}
public class WriteLock : BaseLock
{
public WriteLock(ReaderWriterLockSlim locks)
: base(locks)
{
Locks.GetWriteLock(this._Locks);
}
public override void Dispose()
{
Locks.ReleaseWriteLock(this._Locks);
}
}
Thursday, January 07, 2010
UPDATE: Thanks for all those who commented on this. Turns out the problem was pretty simple. You just have to get rid og the ‘FileId’ and ‘DirectoryId’ properties. They’re ONLY for navigation so cannot be in the entity as fields too…easy peasy :)
I like EF, conceptually and in practice it’s pretty nice. EF4 adds a lot of the missing goodness. There’s one thing which ALWAYS stumps me in EF 3 though…this VERY simple paradigm…
Given this class layout (generated from a SQLLite DB)
See the little 0..1->* relationship? Should be dead simple, the DirectoryId in the UploadFile entity maps to the WatchDirectory Entity…with this association:
I ask you, how the hell can that fail??? But no matter what, this happens…
Error 3 Error 3007: Problem in Mapping Fragments starting at lines 124, 154: Non-Primary-Key column(s) [DirectoryId] are being mapped in both fragments to different conceptual side properties - data inconsistency is possible because the corresponding conceptual side properties can be independently modified.
G:\Work\cozwecan_code\UploadClient\cozwecan.pix.client\cozwecan.pix.client\Model.edmx 155 11 cozwecan.pix.client
Anyone got more of a clue about EF than I have???
Thought I’d just post some dumb little utility classes that I had hanging about in my code repository (and always end up trying to find).
They’re just little utility classes for finding out image format / extensions / content types for various images…(and before any smartass pipes up, yes much of this info is in registry…but this is intended for use where I can’t access that)
public static class Imaging
{
public static string GetContentTypeByImageFormat(ImageFormat format)
{
string ctype = "image/x-unknown";
if (format.Equals(ImageFormat.Gif))
{
ctype = "image/gif";
}
else if (format.Equals(ImageFormat.Jpeg))
{
ctype = "image/jpeg";
}
else if (format.Equals(ImageFormat.Png))
{
ctype = "image/png";
}
else if (format.Equals(ImageFormat.Bmp) || format.Equals(ImageFormat.MemoryBmp))
{
ctype = "image/bmp";
}
else if (format.Equals(ImageFormat.Icon))
{
ctype = "image/x-icon";
}
else if (format.Equals(ImageFormat.Tiff))
{
ctype = "image/tiff";
}
return ctype;
}
public static ImageFormat GetImageFormatByContentType(string contentType)
{
ImageFormat format = null;
if (contentType != null)
{
if (contentType.Equals("image/gif"))
{
format = ImageFormat.Gif;
}
else if (contentType.Equals("image/jpeg") || contentType.Equals("image/pjpeg"))
{
format = ImageFormat.Jpeg;
}
else if (contentType.Equals("image/png"))
{
format = ImageFormat.Png;
}
else if (contentType.Equals("image/bmp"))
{
format = ImageFormat.Bmp;
}
else if (contentType.Equals("image/x-icon"))
{
format = ImageFormat.Icon;
}
else if (contentType.Equals("image/tiff"))
{
format = ImageFormat.Tiff;
}
}
return format;
}
public static string GetFileExtensionByContentType(string contentType)
{
string ext = "bin";
if (contentType.Equals("image/gif"))
{
ext = "gif";
}
else if (contentType.Equals("image/jpeg") || contentType.Equals("image/pjpeg"))
{
ext = "jpg";
}
else if (contentType.Equals("image/png"))
{
ext = "png";
}
else if (contentType.Equals("image/bmp"))
{
ext = "bmp";
}
else if (contentType.Equals("image/x-icon"))
{
ext = "ico";
}
else if (contentType.Equals("image/tiff"))
{
ext = "tif";
}
return ext;
}
public static string GetContentTypeByFileExtension(string fileExtension)
{
switch (fileExtension)
{
case ("gif"):
return "image/gif";
case ("jpg"):
return "image/jpeg";
case ("jpeg"):
return "image/jpeg";
case ("bmp"):
return "image/bmp";
case ("tif"):
return "image/tiff";
}
return "application/octet-stream";
}
}
Friday, December 18, 2009
Well, I got a job with a company in Glasgow just after I returned…quit today…various reasons why I left and I don’t want to rake over old ground. I’m working on my resume…so…first bit, Job history. Never quite sure how much detail to put in here…just a list of titles and dates / paragraph about each one…whaddya think?
Word version: here
Scott Galloway
Phone Number: 07563922053
Blog: http://www.mostlylucid.net
Email: scott.galloway@gmail.com
DOB: 02/02/1973
Work History
· 01/2007-10/2009 – Microsoft Corporation, Redmond, WA
Program Manager, ASP.NET Team / Project Server Team
I was responsible for the design of Product features; these included the design of all WebForms features in ASP.NET 4 and potential future releases. I was also responsible for managing the release process of a wide range of products including ASP.NET MVC.
· 06/2005-01/2007 – Microsoft Limited, Reading, UK
Application Development Consultant
As an ADC I advised a number of businesses on the improvement of both application design and architecture as well as improving development practices. I specialized in ASP.NET and Sharepoint as well as in the performance and scalability of enterprise web applications.
· 02/2003-06/2005 – Storm ID Ltd, Edinburgh, UK
Senior Software Architect
At StormId I was involved in the full lifecycle development of a number of customer facing Web Sites. These sites were both ASP.NET as well as classic ASP and SQL Server based.
· 03/2002-02/2003 – VisitScotland.com, Edinburgh, UK
Technical Architect (management responsibilities)
Whilst at VisitScotland.com I oversaw the migration of a Java based Corba architecture to a J2EE / JSP and patterns based system. Additionally I mentored a team of classic ASP developers in the building of a number of high scalability customer facing websites.
· 06/2000-03/2002 – GlobalFarmers.com, Edinburgh, UK
Internet Development Manager(management responsibilities)
Globalfarmers.com was the largest internet startup in Scottish history. I was brought in to bring development back on track with a third party supplier. The system was built using J2EE and an OODMBS, I also developed a subsidiary PHP based system.
· 02/1999-06/2000 – BlackID, Glasgow, UK
Web Developer
At BlackID I worked on the server side elements of web sites for huge customers such as Burger King.
Tuesday, October 13, 2009
Tomorrow, after 4 1/2 years, including 2 years with the ASP.NET Team I’m finally leaving Microsoft.
The reasons for leaving are varied, I am resigning, but the timing is not entirely my own choice. I’m looking at this as an opportunity, I’ve done a huge amount of thinking about the course my life should take next and this really is a positive step in that direction. I have no bitterness nor animosity towards Microsoft, they hung in longer than a lot of employers would and that led me to stay too long in a job I wasn’t suited for. They are a great company, the majority of the people I worked with are passionate, really smart people and I’ll miss them a lot. Some others are power hungry jerks and completely dissociated from the real world…but then every company has their share of those people.
I am at heart a developer, I love designing and building web applications and frankly I’m incredibly good at it and I especially love working with other people who are good at it too (or want to be). I posted some time ago about how I got to where I was (well, am until about midday tomorrow). Now I need to look for a job which gives me the same level of excitement which I’ve lost of late. I am really grateful to have had the chance to mature professionally, all the prima-donna shit which comes through lacking confidence in my own skills that I had previously has gone. I’ve seen what life is like in the place every developer using ASP.NET longs to go and well, it wasn’t for me.
Right now I plan to move back to my native Scotland and find a job where they need someone like me, someone who still has a passion for web technologies and the drive and knowledge to build pretty much anything.