web.config name casing issue fixed

OK, after some teething pains, the new box is up and running, and I have fixed the web.config casing issue - the one where Umbraco was expecting a lower cased name, however, where mono would accept web.config or Web.config.

Of course, this issue would never have hit file systems that are case insensitive (e.g. default windows installations). But in our case, it created issues: when the web configuration file was called 'Web.config', Umbraco has difficulty finding it, and also re-generated 'web'.config' files in the Views folder.

So I wrote some code so that if we do not find 'web.config', we look for 'Web.config', and then if that is not found as well, we either re-generate, or throw an error.

On another note, on the new box, I could not get the monodevelop NUnit plugin to work. It runs for exactly 170 tests and then the UI crashes. I suspect a glibc compatibility issue. But to stay on track, I ll ignore this for the time being. I was able to run tests with the NUnit 2.6.3 gui application.

New Linux box is up

Yes, the new Linux box is finally up and I have set up the parallel mono development environment as well for mono 3.0.11. I am ready to sort out the casing issue that affected the web.config file.

Setting up the new system required that I re-set up NuGet. I do not think I have discussed this before. This is discussed in detail here. To briefly summarise (assuming /opt/mono as install location),

$ mkdir /opt/mono/local
$ mkdir /opt/mono/local/bin
$ mkdir /opt/mono/local/bin/NuGet
$ cd /opt/mono/local/bin/NuGet
$ wget http://download-codeplex.sec.s-msft.com/Download/Release?ProjectName=nuget&DownloadId=741331&FileTime=130256444329670000&Build=20798
$ cd /opt/mono/bin

$ nano -w nuget

(add the below and save)
#!/bin/sh
exec mono --runtime=v4.0.30319 --gc=sgen /opt/mono/local/bin/NuGet/NuGet.exe "$@"

There is also a really bizarre issue with the Monodevelop parser, and it can get confused parsing .aspx files. The quick hacky solution is to add a variable (control) declaration to the designer.cs file:

src/Umbraco.Web.UI/install/steps/Skinning/loadStarterKitDesigns.ascx.designer.cs View file @ 102a3fe
@@ -2,5 +2,6 @@
   
     public partial class LoadStarterKitDesigns {
+    protected global::System.Web.UI.WebControls.Panel pl_CustomizeSkin;
     }
 }

(based on)
<%@ Control Language="C#" AutoEventWireup="True" CodeBehind="LoadStarterKitDesigns.ascx.cs" Inherits="Umbraco.Web.UI.Install.Steps.Skinning.LoadStarterKitDesigns" %>
<%@ Import Namespace="umbraco.cms.businesslogic.packager.repositories"  %>

<asp:PlaceHolder ID="pl_loadStarterKitDesigns" runat="server">
<asp:Panel id="pl_CustomizeSkin" runat="server" Visible="false">
...

That's it for now.

#! CrunchBang...

OK, I have been setting up a new development box with CrunchBang Linux - a light-weight Debian based linux package.

Setting up for mono 3.0.6 requires pulling in the unstable Debian sources. Setting up the parallel development environment was also fairly straightforward. I cloned mono, xsp, and libgdiplus from the mono project on github. The configuration required the installation of a few additional packages, but the build bootstrap with autogen.sh gave good error messages as to what might be missing.

Contrary to the ArchLinux test box, this one runs on kernel 3.2, and I will also set up a local Apache and nginx server within the parallel mono environment.

6.0.6 mono release delay

I was hoping to release the mono 6.0.6 version this Sunday, however, after having upgraded to mono 3.2.3, I started getting a strange error: it said the "ClientDependency.Core.CompositeFiles.CompositeDependencyHandler" could not be found. In reality, it was there but mono could not load the type.

I subsequently downgraded mono to 3.2.1, and then to 3.0.11. However, this time xsp broke.

Hence, I have decided to delay release until I know for sure what is going on, and this error is resolved.

On the plus side, I have fixed one more issue and identified a new one:

• Fixed: Issue with SystemDirectories.Root being "" - mono expects a rooted value,
  in the virtual path utility: VirtualPathUtility.cs 271.


• Identified: The umbraco code base makes reference to 'web.config' in ApplicationContextExtensions.cs, PluginViewEngine.cs, RenderViewEngine.cs, MediaUploader.ashx.cs, BaseSeleniumTest.cs. Mono can handle both 'web.config' and 'Web.config', but I will need make sure that we have the same behaviour in the Umbraco code base.

uComponents in mono

The stock uComponents 5.4.1 package will not work in mono.

But we love this package and it is one of the best and most useful packages available for use with the Umbraco CMS.

Fortunately, the fixes required to compile uComponents in mono are very small. And here they are (all very minor casing issues):

$>hg diff

diff -r 510ecd773bcb uComponents.DataTypes/DataTypeGrid/Configuration/DtgConfiguration.resx
--- a/uComponents.DataTypes/DataTypeGrid/Configuration/DtgConfiguration.resx	Thu Mar 14 20:11:24 2013 +0100
+++ b/uComponents.DataTypes/DataTypeGrid/Configuration/DtgConfiguration.resx	Fri Sep 20 22:56:39 2013 +0100
@@ -119,6 +119,6 @@
   
   
   
-    datatypegrid.config;System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;utf-8
+    DataTypeGrid.config;System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;utf-8
   
 
\ No newline at end of file
diff -r 510ecd773bcb uComponents.DataTypes/DataTypeGrid/WebServices/DtgWebServices.resx
--- a/uComponents.DataTypes/DataTypeGrid/WebServices/DtgWebServices.resx	Thu Mar 14 20:11:24 2013 +0100
+++ b/uComponents.DataTypes/DataTypeGrid/WebServices/DtgWebServices.resx	Fri Sep 20 22:56:39 2013 +0100
@@ -119,6 +119,6 @@
   
   
   
-    prevaluewebservice.asmx;System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;utf-8
+    PreValueWebService.asmx;System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;utf-8
   
 
\ No newline at end of file
diff -r 510ecd773bcb uComponents.DataTypes/Shared/AjaxUpload/AjaxUploadHandlerResource.resx
--- a/uComponents.DataTypes/Shared/AjaxUpload/AjaxUploadHandlerResource.resx	Thu Mar 14 20:11:24 2013 +0100
+++ b/uComponents.DataTypes/Shared/AjaxUpload/AjaxUploadHandlerResource.resx	Fri Sep 20 22:56:39 2013 +0100
@@ -119,6 +119,6 @@
   
   
   
-    ajaxuploadhandler.ashx.txt;System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;utf-8
+    AjaxUploadHandler.ashx.txt;System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;utf-8
   
 
\ No newline at end of file
diff -r 510ecd773bcb uComponents.DataTypes/Shared/WebServices/SharedServices.resx
--- a/uComponents.DataTypes/Shared/WebServices/SharedServices.resx	Thu Mar 14 20:11:24 2013 +0100
+++ b/uComponents.DataTypes/Shared/WebServices/SharedServices.resx	Fri Sep 20 22:56:39 2013 +0100
@@ -119,6 +119,6 @@
   
   
   
-    dictionaryservice.asmx;System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;utf-8
+    DictionaryService.asmx;System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;utf-8
   
 
\ No newline at end of file
diff -r 510ecd773bcb uComponents.DataTypes/UrlPicker/Services/UrlPickerServiceResource.resx
--- a/uComponents.DataTypes/UrlPicker/Services/UrlPickerServiceResource.resx	Thu Mar 14 20:11:24 2013 +0100
+++ b/uComponents.DataTypes/UrlPicker/Services/UrlPickerServiceResource.resx	Fri Sep 20 22:56:39 2013 +0100
@@ -119,6 +119,6 @@
   
   
   
-    urlpickerservice.asmx;System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;utf-8
+    UrlPickerService.asmx;System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;utf-8
   
 
\ No newline at end of file
diff -r 510ecd773bcb uComponents.DataTypes/uComponents.DataTypes.csproj
--- a/uComponents.DataTypes/uComponents.DataTypes.csproj	Thu Mar 14 20:11:24 2013 +0100
+++ b/uComponents.DataTypes/uComponents.DataTypes.csproj	Fri Sep 20 22:56:39 2013 +0100
@@ -357,7 +357,7 @@
     
     
     
-    
+    
     
     
     
@@ -668,4 +668,4 @@
       
     
   
-
\ No newline at end of file
+
diff -r 510ecd773bcb uComponents.sln
--- a/uComponents.sln	Thu Mar 14 20:11:24 2013 +0100
+++ b/uComponents.sln	Fri Sep 20 22:56:39 2013 +0100
@@ -43,14 +43,10 @@
 		Settings.StyleCop = Settings.StyleCop
 	EndProjectSection
 EndProject
-Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "uComponents.Core.UnitTests", "uComponents.Core.UnitTests\uComponents.Core.UnitTests.csproj", "{35953BAA-8E67-4E27-B4E5-96D142DC924B}"
-EndProject
-Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Unit Tests", "Unit Tests", "{026CA274-604D-49B5-932B-EB88A1E20BF3}"
-	ProjectSection(SolutionItems) = preProject
-		Local.testsettings = Local.testsettings
-		uComponents.vsmdi = uComponents.vsmdi
-	EndProjectSection
-EndProject
+
+
+
+
 Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "uComponents.XsltExtensions", "uComponents.XsltExtensions\uComponents.XsltExtensions.csproj", "{1665551B-1EE8-4A5A-8857-4A002F4DB3B9}"
 EndProject
 Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "uComponents.DataTypes", "uComponents.DataTypes\uComponents.DataTypes.csproj", "{E83250BD-27B0-4B39-90C8-F07538058938}"

One note: There might be some issues with the DataGrid component. But I have not had a close look at this yet.

Site testing update

I have been testing using a production grade MVC (windows) Umbraco site

First impressions: 99% doing OK.

• Subsystem: Mono - 3.2.1, xsp - 3.0.11, nginx, archlinux 3.10.10-1


• After updating Web.config - a manual restart of fcgimono service is needed. This is a critical issue, and probably limits commercial application. Not sure why this happens...


• uComponents 5.4.1: with a few minor tweaks this runs - but there might be some issues with the DataGrid control. More details coming soon.


• Ninject: In Mono requires Ninject.Mono 3.0.1.13 or greater (from NuGet). The usual Ninject package will not work.


• Porting of windows Umbraco apps requires careful consideration of casing. This should not be an issue for apps developed natively in Linux / mono.

Mono versions and fast cgi

A quick note on mono versions and fastcgi-mono-server4

These versions seems to work together well: (With 2.11 being outdated, and in fact never having been released, I would recommend the 3.0.x series. I am using 3.0.7 at present.)

Mono 2.11.4 - git 1723480147411f3973ebceba66f518db69131d10
xsp - git 6365230e6ef348045078f477af085429dceedb3d

File: /etc/systemd/system/fcgimono.service  
[Unit]
Description=Mono fcgi Server
Before=nginx.service
After=nss-user-lookup.target

[Service]
EnvironmentFile=
Type=simple
ExecStart=<mono_path>/fastcgi-mono-server4 \
        --applications=*:<port>:/:/<app_path> \
        --socket=unix --filename=/tmp/fcgimono.sock --verbose=true \
        --loglevel=All
User=http
Group=http

[Install]
WantedBy=multi-user.target

Mono 3.0.6 - git 20e40c448cedb603f8c1bdf2b29ffc4d43b721b6
Mono 3.0.7 - git 514fcd79208764c170851c61498d650dc357b429
Mono 3.0.11 - git 6df55cf92a314af75c2723ff9a1f3c4128c52dfd
xsp - git 4587438369691b9b3e8415e1f113aa98b57d1fde

File: /etc/systemd/system/fcgimono.service  
(as above)

The below DOES NOT work, and serves blank pages. Might be a simple fix
but I have not looked at it yet.
Mono 3.2.1 - git f3f789e7f826ea9077564c35a0a6e38f09e4c664
xsp - git ae70e0355edf5ac91cb9aa81a0d1075cda62c384

File: /etc/systemd/system/fcgimono.service  
(note below syntax change)
ExecStart=/fastcgi-mono-server4 \
        /applications=*:8083:/:/ \
        /socket=unix  /filename=/tmp/fcgimono.sock /verbose=true \
        /loglevel=All

One final note - if you are installing on a parallel development environment, then under some circumstances, the path the the fastcgi executable is not built correctly. The below example demonstrates:

File: /opt/mono/bin/fastcgi-mono-server4 
#!/bin/sh
 - exec /usr/bin/mono $MONO_OPTIONS "/opt/mono/lib/mono/4.5/fastcgi-mono-server4.exe" "$@"
 + exec /opt/mono/bin/mono $MONO_OPTIONS "/opt/mono/lib/mono/4.5/fastcgi-mono-server4.exe" "$@"

Oh, and before I forget:

The parallel environment file needs export statements 
(even though systemd complains) or else the web app breaks. 
Example:

File: /opt/mono/etc/mono_dev-env 
#!/bin/bash
MONO_PREFIX=/opt/mono
MONO_OPTIONS="--debug"
export DYLD_LIBRARY_FALLBACK_PATH=$MONO_PREFIX/lib:$DYLD_LIBRARY_FALLBACK_PATH
export LD_LIBRARY_PATH=$MONO_PREFIX/lib:usr/lib
export C_INCLUDE_PATH=$MONO_PREFIX/include:/usr/include
export ACLOCAL_PATH=$MONO_PREFIX/share/aclocal:/usr/share/aclocal
export PKG_CONFIG_PATH=$MONO_PREFIX/lib/pkgconfig:/usr/lib/pkgconfig
export PATH=$MONO_PREFIX/bin:$PATH
export MONO_GAC_PREFIX=$MONO_PREFIX:/usr
PS1="[mono] \w @ "

if you are going to be switching environments, then these variables need to be 
reset. Like so:

File: /etc/profile.d/mono.sh
MONO_PREFIX=/usr

#unset parallel mono environment
export MONO_GAC_PREFIX=$MONO_PREFIX
export DYLD_LIBRARY_FALLBACK_PATH=`echo ${DYLD_LIBRARY_FALLBACK_PATH} | awk -v RS=: -v ORS=: '/opt/ {next} {print}$
export LD_LIBRARY_PATH=`echo ${LD_LIBRARY_PATH} | awk -v RS=: -v ORS=: '/opt/ {next} {print}'  | sed 's/:*$//'`
export C_INCLUDE_PATH=`echo ${C_INCLUDE_PATH} | awk -v RS=: -v ORS=: '/opt/ {next} {print}'  | sed 's/:*$//'`
export ACLOCAL_PATH=`echo ${ACLOCAL_PATH} | awk -v RS=: -v ORS=: '/opt/ {next} {print}'  | sed 's/:*$//'`
export PKG_CONFIG_PATH=`echo ${PKG_CONFIG_PATH} | awk -v RS=: -v ORS=: '/opt/ {next} {print}'  | sed 's/:*$//'`
export PATH=`echo ${PATH} | awk -v RS=: -v ORS=: '/opt/ {next} {print}'  | sed 's/:*$//'`

And a mono / libgdiplus compilation note:

MONO libgdiplus needs giflib 4.x - won't compile with giflib 5.x.
If in a bind, you can compile libgdiplus without gif support.

6.0.6 Tests Update II

I have now gone through all the tests, and am pleased to say that 909 tests are passing, and that testing is concluded.

The most interesting errors are logged in the Umbraco error tracker Incorrect Parse of date..., and here MySql republish all fails.

Otherwise, we had some error fixes due to XPathNodeIterator differences, which are well-known.

The remaining few failures appear to be of technical origin: e.g. string encrypt / decrypt tests fail due to null AppDomainAppVirtualPath handling System.Web.Util/UrlUtils (161) in mono. The mono HttpRuntime is coded and coupled very differently. And, to sort this out, we would need to create a wrapper for the FormsAuthentication... calls

I ll be doing some front-end and back-office tests next.

6.0.6 Tests Update

I fixed partial trusts test, by using a simple medium trust web.config. So now all partial trust and related tests that do not require CAS (Code Access Security) are passing under mono with NUnit 2.6.1

The NUnit version is also significant: NUnit 2.6.2 uses System.Runtime.Remoting.Messaging > LogicalGetData and LogicalSetData during its

TestContext.CurrentContext.Test.Name

calls.
But these are not supported yet in mono. We therefore need to use NUnit 2.6.1

Test Stats: 889 / 1086 passing. Ignored 169. I have about 28 tests left, of which some fail by assertion, e.g. the CAS tests. When all test that can pass are passing, I will go back and run another user test.

6.0.6 Update

Just a quick update, I have been merging in the 6.0.3 changes to 6.0.6. At present, 867 / 1086 tests are passing.

6.0.6 - Compile mark reached

6.0.6 now compiles in mono - it only needed a few casing changes in resource files, and some changes to the SQLCE project (which we do not use in mono - but for safety I am making sure it compiles). I also had to remove comments from some auto generated designer.cs files.

On an unrelated note - if you get this:

System.Security.SecurityException
No access to the given key

It means that the MS .NET version of Microsoft.Web.Infrastructure.dll is present. Removing the MS .NET dll is sufficient to clear this error.

Codegarden 2013 - Quick notes

Codegarden was a great event as usual. We got a glimpse into the changes that are going on with Umbraco version 7 - the big topic was Angular JS.

My mono / Linux presentation proceeded as planned. And I also had a chance to catch up with the core team to discuss how we can bring the mono changes into the Umbraco project.

On other very good news, Umbraco is now hosted on git hub, and this makes life a whole lot easier!

I will continue with the mono development on my git hub project and deliver production ready distributions for 4.7.2, 4.11.x, and 6.0.6. After this is done, there will be no further development in this repo. I anticipate that in 2 - 3 months, this should all be completed.

For those who want a head start, I have already created a sample Linux web site with sample configuration files here. However, this sample uses Umbraco 6.0.3, and therefore is not production ready. It also has a Lucene bug and an intermittent Xml Cache issue, which remain to be fixed. But it is great for a head start.

I also intend to release an ISP guide, which will make it easier for ISPs and hosting providers to create and set-up mono / Linux based Umbraco hosting environments.

Happy computing everyone!

Codegarden 2013 is here

Alright guys, see you all tomorrow...

6.0.3 Tests - IV

With, Codegarden 2013 happening in a few days time, I will be demonstrating an alpha version of the 6.0.3 code. As we know, 6.0.3 suffers from a security flaw, and is not suitable for production systems.

There are two additional caveats:

• Lucene searches report incorrect nesting levels (1 less than the actual)
  


• Some content service tests are also failing and I do not yet know why

But overall 878 test are passing now.

6.0.3 - Tests III

Yay, 840 tests are passing...

Let,s look at the highlights for the passing tests:

• Majority of pass results due to test set-up changes - in particular the Mysql db set-up.
  


• The stock NUnit dll pulled in by NuGet Restore (2.6.2) has isues in mono, and the NUnit introspection code will not work. Resolved by using NUnit 2.6.1 from ArchLinux AUR (required edit of AUR package file). NuGet 2.6.1 should work, but I have not tried it.


• Routing...
  

  RouteTable.Routes.Clear();

  
  Does not appear to work. Replaced with:
  

  for (var i = RouteTable.Routes.Count - 1; i > -1; i--)
  				RouteTable.Routes.RemoveAt(i);

  
  


• DbProvider issue...

  DbProviderFactories.GetFactory(_providerName)

  
  Goes to the machine.config file - so the dbProvider must exist there. Resolved by adding the MySql dbProvider to machine.config. Cause: This is b/c in tests HttpRequest is not defined and in mono this leads to a loss of the path (GetCurrentPath) to the nunit test config file.
  


• Case insensitive Linq style query tests, e.g. (...).level will fail because the existing 'rescue' code fails in mono. To fix this I added:
  

  
  namespace Umbraco.Core.MultiPlatform
  {
  	public class ReflectionHelper
  	{
  		public static BindingFlags GetBindingFlagsCasingSafe(BindingFlags bindingFlags)
  		{
  			if (PlatformHelper.IsMono)
  				return bindingFlags | BindingFlags.IgnoreCase;
  
  			return bindingFlags;
  		}
  	}
  }
  

  
  

---

And, let's look at the highlights for the failing tests:

• 
  Majority in Persistence.Repositories and Service.ContentService tests - may be due to database set-up issues.
  


• PartialTrust...WhenMethodShouldNotSucceed - fail due to CAS implementation issues in mono.
  


• String Extension Tests - encrypt / decrypt fail b/c of mono implementation differences. In particular, the HttpRuntime class is implemented very differently in mono, and I have not yet found a way to mock it. The encryption code leads to a direct call involving the HttpRuntime, and fails whenAppDomainVirtualPath is null. Easiest way to fix this would be to wrap the encryption code to make it mockable. In particular,
  

  FormsAuthentication.Encrypt(...)

  calls.
  

6.0.3 - Tests II

Passing tests are now at 672 of 899 668bac0551. I have added a few ignored tests for the mysql db tests bringing the ignored tests total to 162.

Top failure causes:

• Issues relating to dropping of foreign constraints at umbracodomains
  


• issues relating to test initialisation
  


• Errors due to mono implementation differences.
  

To get the tests to work at this stage, I added a Configuration settings injector - this is because NUnit is not able to find and load the appropriate configuration settings. This is a bigger topic than I can give it justice here.

Overall the test area will need quite a bit of work, if we want to achieve out-of-the-box native testing capability.

6.0.3 - Tests

I have started working on the 6.0.3 tests: commit hash 648b701599. Out of the box 324 / 1061 tests were passing. All of the database tests are (of course being written with SQLCE) failing. The test code itself looks to be in a transitional stage from the sqlHelper based set-up towards using petaPoco.

With a bit of coaxing, I have increased passing tests to 390. The increase comes mainly from non-database based tests. Some of the database based tests are passing in the 'legacy' setting - using the MySqlTestHelper.

I have been going through these tests very quickly as there are quite a lot of them... The testing strategy I am using has required big changes to the underlying source code - i.e. the replacement of all calls to configuration manager - of which there are quite a lot now. I would recommend having a mockable wrapper class for these calls - at some point.

I ll be continuing and looking at the database based tests next.

6.0.3 Update

Ok the 6.0.3 version is working now (git hash 17411e5c8a), and I have tested it for the most basic things. In general, so far so good.

I also started separating all of the 'Multiplatform' code into its own namespace (in Umbraco.Core.MultiPlatform). As a first step everything is in static functions.

However, I have also observed three behaviours, the cause of which I do not know at the moment, and which are problematic. Here's the list:

• EditMemberTypes - renders 'Allow Only at root' in Firefox
  


• The first hit on the root page after 'Publish' gives a 404. Second hit produces the page
  


• Firefox: sometimes hangs and the javascript debugger kicks in
  

I will have a look at these in due course. The next step is to get the tests passing and to run through setting up a sample site. While the UI loads fine and we got basic functionality, there are parts that i have not yet ported; and sorting those out is next. I am hoping that when that is all done, I can have a look at the above snagging list.

6.0.3 UI Load mark reached

Yay! As of 80c6589fec, the 6.0.3 UI is now loading. Let's hit the highlights:

• Application Initialization: CacheHelperApplicationEventListener and HttpContext.Current are initialised twice during start up and this breaks initialisation. I wrote some workaround code for now. I think there may be some mono differences during application start up or some threading issues. This will need a closer look. However, at first look the app startup code does look possibly too heavy.


• This setting breaks mysql server initialisation in Linux:
  

  lower_case_file_system=1

  
  The setting does not work in linux / unix as it is 'read only' (here). So I have also disabled this check if we hit a linux / unix platform set-up. We could have kept the check, but this would automatically disable Umbraco mysql install on the majority of Linux systems, which by default have a case sensitive file system set-up. Oh, and adding the setting to the my.cnf file in arch linux causes the mysql daemon start up to fail. So, adding it to the mysql config file may break your server.

That's it for now.

6.0.3 - Started

Finally time for a progress report on 6.0.3 - I have finally started on this and there is now a git branch called 'u6.0.3'. With this change set (1cd0904b19), version 6.0.3 compiles in Monodevelop 4. The overall changes were minimal. However, some preliminaries are required.

Here are the preliminaries:

• Monodevelop: As of today, you will need git rev cfb8e3bc5d or c1cecf5931. Avoid rev b916eecb8c, as this rev does not compile cleanly. We need these changes so that references are updated correctly in the .csproj files.


• Nuget package restore. There is a tool available for this in:
  

  $ tools/mono/RestoreNugetPackages.sh

  
  This is a blunt tool and restores all packages, including Microsoft.Web.Infrastructure, which we actually do not want, as we will be using Mono's corresponding dll. So project references need to re-adjusted afterwords.


• The shell script above uses a small custom build task. I was only able to write this as a C# assembly. I could not write it as an inline build task, as xbuild was throwing errors. The build task itself fixes another xbuild bug, namely incorrect handling of the ';' and '\' character. The former is always interpreted in the unix context, and the latter is always converted to '/'. Related info here.


• Finally, as Mono does not yet fully support native MVC 4, I have switched to using the stock ASP.NET stack that we get from NuGet. We will see what kind of mileage we will get from these.

4.11.3 - Tests II

478 / 499 (the 500th test does not contain any active tests) tests are passing. Of the remaining 21 tests, cause of failures are:

• 
  Partial Trust: ...(ReportsFailure) - I have not been able to get these to work. Apparently in mono CAS (Code Access Security) is experimental and unsupported. I have not been able to enforce Reflection Access restrictions in test code, nor have I been able to debug it in an NUnit test instance. Please see here for details.
  
  In mono it appears that,
  

  
  SecurityManager.SecurityEnable returns false unless 
  mono is run with the --security option. 
  

  
  


• 
  Published Content: DynamicNode, DynamicPublishedContent, DynamiXml, StronglyTypedQuery tests - Not sure yet if failure is because of test set-up differences or a genuine implementation difference in the .NET mono implementation.
  


• 
  Published Content: DynamicNode Get_Member_Property fails because it needs some additional test database initialisation, namely the alias of the parent document type. (TO DO)
  

Summary: All is looking good but both the Partial trust and DynamicNode / PublishedContent / Xml tests indicate that there may be some issues due to implementation differences. This will need further clarification and testing.

Git commit hash a1f5565762

4.11.3 - Tests

The Umbraco HQ team has done a great job with the tests, and the testing architecture. I have done a few small changes, e.g. moving the test database over to MySql by way of an MySqlTestHelper.

At present about 269 /500 tests are passing. Some of the failing tests pass when run individually. This has been a slow laborious process, and Monodevelop has been frequently freezing after failing tests.

I will need to update my own test (4.7.2) code at some point. I am using it here but it is not a good architectural match, and will need to be refactored.

For compatibility it might be good in the future to move the test database to SqlLite - or another db that can be used in Windows and non-Windows environments.

4.11.3 - Of Razor, Mvc, and More

I have started testing Razor and Mvc features. Here are the important bits:

• XPathNodeIterator MoveNext issues revisited:
  

  Example:
  if (media != null && media.Current != null) 
  {
      media.MoveNext();
      ...
  }
  

  
  Does not work in mono, as until MoveNext() is called Current is always null.
  


• 'Could not locate Razor Host Factory type: umbraco.MacroEngines.RazorUmbracoFactory, umbraco.MacroEngines' error message - this stems from a bug in mono. The quick resolution is to use either one of the two below:
  

  
  In web.config use:
    <system.web.webPages.razor>
      <host factoryType="umbraco.MacroEngines.RazorUmbracoFactory" />
      
  or use
    <system.web.webPages.razor>
      <host factoryType="umbraco.MacroEngines.RazorUmbracoFactory, umbraco.MacroEngines, Version=1.0.4832.22841, Culture=neutral, PublicKeyToken=null" />
  

  
  The 'bug' is in the mono type enumerator:
  

  
  In mono System.Web.Compilation > BuildManager > public static Type GetType (string typeName, bool throwOnError, bool ignoreCase)
  
  ...
  	var aname = new AssemblyName (typeName.Substring (comma + 1));
  	wantedAsmName = aname.ToString ();
  ...
  	if (String.Compare (wantedAsmName, asm.GetName ().ToString (), StringComparison.Ordinal) == 0) { 
  ...
  

  
  Looks for an exact match in assembly name if one is supplied
  


• Some razor logic seems to produce a compiler error:
  

  
  @inherits umbraco.MacroEngines.DynamicNodeContext
  @{ 
  	var level = String.IsNullOrEmpty(Parameter.Level) ? 1 : int.Parse(Parameter.Level); 
  	var ulClass = String.IsNullOrEmpty(Parameter.UlClass) ? "" : String.Format(" class=\"{0}\"", Parameter.UlClass); 
  	var parent = @Model.AncestorOrSelf(level);
  	if (parent != null) { ** fails here **
           ....
  	}
  }
  

  
  Produces 'single file build failed,' and this can be traced back to a lock in the mono System.Web.Compilation > BuildManager.cs > static void Build (VirtualPath vp) > line 404.
  

  
  ...
          static readonly object bigCompilationLock = new object ();  
  ...
  	CompilationSection cs = CompilationConfig;
  	lock (bigCompilationLock) {
  

  
  This is the most serious error so far, and needs some investigation. However, in this case, removing the null check is sufficient to make the compilation succeed, and workarounds therefore, may exist.
  


• Summary 1: in addition to casing issues, the XPathNodeIterator MoveNext issue is emerging as a key issue.
  


• Summary 2: Razor and MVC functionality will need further testing.
  

4.11.3 - Content Area

OK, the Content area is now finished as well. All superficial UI functions have now been fixed. The latest round of fixes address some important mono differences:

• 1f54a0c35f - Our good friend: To initialise an XPathNode Iterator, call MoveNext() first. Content > Public Access was throwing errors, which were fixed with MoveNext calls (via the introduction of an XmlHelper method).


• 7ee981a56b - The mono multiline text box control tab injection bug was fixed by some jQuery


• 3b7f42c97b - The false positive in the xss check when first logging in was fixed as well. This error originated because mono treats absolute paths starting with '/' as a file system path (in the Uri class), and sets the Host name to an empty string.

4.11.3 Users Area

The users area is now completed as well. Again, this was pretty much smooth sailing, except that in the edit user type section, the dynamic checkbox list was losing its state. Again, I applied a local fix. So the dynamic checkbox state issue is revealing itself to be quite significant.

I have also started on the Content area. More on that in the next post.

4.11.3 - Update

Well, I have now also cleared the Developer > Packages area, and also the Media, and Member areas, which are in working order for all basic tasks.

There were mostly casing changes, but there are a couple of important points to report - both on the Developer > Packages section:

• During the package uninstall dialog, the dynamically created checkboxes lose state, and this needed fixing. This is a well known mono issue and I have applied the same fix for it in three different places - a bit of repetition, I know. These will be merged at some point. Probably with the 6.0 port.
• The packages callback from umbraco.org returns an all lower-case url, and I changed the casing of the umbraco\developer\Packages\ folder the 'packages'. I generally try to avoid file system changes. But in this case, it was needed.

The packages area functionality will be a difficult to get working - it looks like this area will be most sensitive to platform specific changes such as casing, the directory separator, and also the issue of the casing of xml nodes names (which I have yet to address).

4.11.3 Developer area - UI load errors fixed

OK I have fixed up all the errors experienced during superficial load, and item creation (except Packages). Again this was mostly smooth sailing.

git commit hash f61901a0cc

4.11.3 Settings area cleared

Yes, I cleared the 'Settings' area today. Most of it was smooth sailing: the majority of the changes were casing related, and a few other issues, that I had already fixed in 4.7.2, and now ported over into the new codebase.

There was one small issue of note - that was a new addition: Mono does not implement

AllInternalsVisible=true

And, using this in the attribute declaration will actually make the friend assembly not see the desired internal assembly.

One exception: mono tab bug fix has been deferred for now.

That's it for now.

git commit hash b0b7d974

CMS admin UI now loads

As of commit hash 2ee48f2be9 the CMS admin UI now loads successfully without any errors. Yay!

However, CMS functionality is at present not available. I have been merging in the 4.7.2 code; and on the plus side, up to now, the same fixes have been sufficient to get things working. I will keep up this plan. On the minus side, the 4.7.2 fixes were not intended to be permanent solutions, and should at some point be re-factored.

The automated build scripts work (when you build twice in succession), but are in need of re-writing, but it is not a priority at this point.

There are a number of recurring themes, which will need a closer look - but this is unlikely to happen until after I have completed the version 6 port. Here are some of those recurring themes:

• '~' in paths: mono does not like these in file system paths. The MultiPlatform IO Helper side steps this issue for now. The mono MVC router appears to be able to deal with '~' in web paths, but that discussion is for another time.


• In mono, creating an absolute Uri from a path that begins with a '/' will have its Host name set to an empty string - hence the login redirect check will fail. I have not addressed this yet.


• The Current property of a mono XPathNodeIterator is uninitialised until MoveNext() is called. So accesing this property without calling MoveNext() will always throw an error.


• The bigger issue of whether to use the same or separately branched code bases for the windows and linux / unix versions remains to be dealt with.

Install Sequence now working

Well, this has turned into a fortnightly update... But the install sequence now works and lets us install to a MySql database. The changes to achieve this were minimal. There were mostly casing changes. Commit hash: 45258f2

There is one other significant change - though this may prove to be temporary, but it is this one:

src/Umbraco.Core/ObjectResolution/ManyObjectsResolverBase.cs

From:
if (InstanceTypes.Contains(value))
{
		throw new InvalidOperationException("The Type " + value + " already exists in the collection");
};
InstanceTypes.Add(value);

To:
if (!InstanceTypes.Contains(value))
	InstanceTypes.Add(value);          

This was always throwing an error during boot as one key (which I have forgotten) was being added twice. But it also seems odd to throw an error before it actually occurs. The change gets us moving for now. (But might be worthwhile to look into why this is happening.)

Looking ahead at the changes required for logging in, it looks like the usual suspects are involved: for example, path mapping issues involving the "~" character, xpath issues involving referencing the Current property of an XPathNodeIterator object before calling MoveNext(), and some casing issues.

XBuild now working with 4.11.3

OK, I got xbuild working now.

However, this is an early version, and you will need to compile twice. This is owing to circular references, which I have not yet sorted out. The easiest way to compile branch 4.11.3 is to use commit #b71c7eb with:

cd build
../tools/mono/ClearLeftOverFiles (if needed)
source ~/mono_dev-env (remember to switch to mono dev environment >= 2.11)
xbuild Build.mono.proj /p:Configuration=Debug

This will create a copy of the web application under the usual location _BuildOutputs/WebApp/. You can then run the web app with:

cd _BuildOutputs/WebApp
source ~/mono_dev-env (remember to switch to mono dev environment >= 2.11)
MONO_OPTIONS="--debug" xsp4

Of course, you will immediately get an error: System.InvalidOperationExceptionThe Type Umbraco.Web.CacheHelperExtensions+CacheHelperApplicationEventListener already exists in the collection. I will deal with this in next week's post. But for anybody who wants to get a head start, here is a neat trick, which I recently learned:

source ~/mono_dev-env (remember to switch to mono dev environment >= 2.11)
# open the umbraco solution in monodevelop
MONO_OPTIONS="--debug" MONODEVELOP_SDB_TEST="true" monodevelop &
# in monodevelop: Run > Run With > Custom Command Mono Soft Debugger > Listen
cd _BuildOutputs/WebApp
MONO_OPTIONS="--debug --debugger-agent=transport=dt_socket,address=127.0.0.1:10000" xsp4

Using the above, you can step through and debug the project code. This is the equivalent of attaching to a process in Visual Studio.

Happy computing...

Umbraco 6 is out! Mono roadmap

Yes, Umbraco 6 is finally out! I am already planning the mono port - though the challenges here will be of a greater degree as we may run into issues in mono around MVC4 and the support of asynchronous calls. In practical terms, I plan to focus on the 4.11.3 port in the next 3-4 months, while also doing some research on the best way to proceed with U 6. This was just a quick update note, and also to say thank you to the Umbraco team and community for getting 6 ready and running, in a very short time. Happy Umbraco everyone!

Umbraco 4.11.3 on mono update

Umbraco 4.11.3 now compiles

Umbraco 4.11.3 now compiles in the Monodevelop IDE under mono 3.x. Thanks to the work of the Umbraco HQ team, the changes required were actually quite small: a few casing changes, increasing .NET to 4.5 so that we can use native mono MVC 3, and a few small idiosyncrasies.

Details here: https://github.com/cankoluman/umbracoMono/tree/u4.11.3

Umbraco 4.11.x mono port kicks off

I have had a start on the Umbraco 4.11.x port to mono

4.11.x introduces MVC type views, and can be seen as a stepping stone to Umbraco 6

With the addition of MVC, my quick impression at this point is that mono & monodevelop are not yet quite ready to handle the port "out of the box." Yet, I think we will be able to make a port, but I think it will initially be a bumpy ride.

A number of porting avenues are available

1. Use Visual Studio & mono windows and mono tools


2. Use native (e.g. Linux) mono & tools but with the Umbraco supplied MVC and related dlls


3. Develop on a fully native platform, and use the open sourced asp.net stack that is now also used by mono

1. is probably the easiest way to get started. Especially, if the code is placed on a Linux VM, and shared into the windows host - this will also provide "casing" checks, and can be easily tested in two environments. On the downside, it requires a Windows and VS license (especially if you wanted to use Resharper etc.)

2. is a middle of the road approach. It might side-step the following issue: in mono, MVC 3 is only available for .NET 4.5 and above. But a downside is that, if there are any exceptions originating in the MS dll's we will not be able to debug through these.

3. is the hardest option. It will require immediate changes to all solution projects so that they are incremented to .NET 4.5. At present, there is an annoying bug in the monodevelop IDE and projects referencing MVC3 need to be re-referenced every time the solution is re-loaded.

For people looking to get their hands dirty, I would recommend option 1. Personally, however, I have been using option 3 - mainly because I am interested in developing a platform independent development environment - i.e. one that does not require ownership of a windows / VS license.

Some gottchas for option 3

• If you are compiling mono (and you will be...), at present you need to use automake version < 1.13 so that you still have support for "Cygnus style trees."


• You will need to set up a parallel mono development environment


• Compiling monodevelop is not easy at all. Here's what I have been doing:
  

  
  cd monodevelop
  ./configure
  make 
  find -name '*.csproj' -print -exec sed -i.bak 's/False<\/SpecificVersion>/True<\/SpecificVersion>/g' {} \;
  #find -name '*.csproj' -print -exec sed -i.bak 's/Net_2_0/Net_4_0/g' {} \;
  ./configure --prefix=/opt/mono
  make
  sudo make install
  #switch to mono 3 environment
  (batch file to set environment variables)
  make
  sudo make install
  
  #this also seems to work
  #find -name '*.csproj' -print -exec sed -i.bak 's/False<\/SpecificVersion>//g' {} \;
  

  
  


• With this approach, the Microsoft, "xdt:" config file transforms do not work, and I am looking for an alternative.


• the manual build targets need to be re-written to account for mono's limited implementation of msbuild.

Snapshots are available here, but unless tagged they are not considered to be working: umbraco-mono on git, umbraco-mono on codeplex

Mono parallel environment set-up example

$ cat mono_dev-env 
#!/bin/bash
MONO_PREFIX=/opt/mono
export DYLD_LIBRARY_FALLBACK_PATH=$MONO_PREFIX/lib:$DYLD_LIBRARY_FALLBACK_PATH
export LD_LIBRARY_PATH=$MONO_PREFIX/lib:usr/lib
export C_INCLUDE_PATH=$MONO_PREFIX/include:/usr/include
export ACLOCAL_PATH=$MONO_PREFIX/share/aclocal:/usr/share/aclocal
export PKG_CONFIG_PATH=$MONO_PREFIX/lib/pkgconfig:/usr/lib/pkgconfig
export PATH=$MONO_PREFIX/bin:$PATH
export MONO_GAC_PREFIX=$MONO_PREFIX:/usr
PS1="[mono] \w @ "

script for resetting to regular mono development environment

$ pwd
/etc/profile.d
$ cat mono.sh 
MONO_PREFIX=/usr

#unset parallel mono environment
export MONO_GAC_PREFIX=$MONO_PREFIX
export DYLD_LIBRARY_FALLBACK_PATH=`echo ${DYLD_LIBRARY_FALLBACK_PATH} | awk -v RS=: -v ORS=: '/opt/ {next} {print}'  | sed 's/:*$//'`
export LD_LIBRARY_PATH=`echo ${LD_LIBRARY_PATH} | awk -v RS=: -v ORS=: '/opt/ {next} {print}'  | sed 's/:*$//'`
export C_INCLUDE_PATH=`echo ${C_INCLUDE_PATH} | awk -v RS=: -v ORS=: '/opt/ {next} {print}'  | sed 's/:*$//'`
export ACLOCAL_PATH=`echo ${ACLOCAL_PATH} | awk -v RS=: -v ORS=: '/opt/ {next} {print}'  | sed 's/:*$//'`
export PKG_CONFIG_PATH=`echo ${PKG_CONFIG_PATH} | awk -v RS=: -v ORS=: '/opt/ {next} {print}'  | sed 's/:*$//'`
export PATH=`echo ${PATH} | awk -v RS=: -v ORS=: '/opt/ {next} {print}'  | sed 's/:*$//'`