Statistics & Lies

Light on Statistics / Short on Lies

Bix Updates: Mixin via Cloning and Fody

| Comments

Bix now supports C# interface-based mixins on the master branch. I am currently working on a feature branch where the mixer functionality has been spun off into a new project called Bix.Mixers.Fody.

That's pretty much what I have to say. Feel free to read on for the long version :-)

Bix Mixin Functionality

At last checkin, my concern with was about field initialization and how to handle constructors. In looking to solve that problem, I couldn't figure out good way to make field initialization (and constructor logic, in general) controllable within the strategy of decompiling a reference assembly and manually creating matching IL.

So I pulled the trigger on a big change, a change that I had been avoiding because of the daunting amount of IL level work and because I just wasn't certain that it would work. There were hard problems to solve. For example, if I'm cloning a method Source.GetFoo(), and its return value is of type Source, then the intent is likely for the cloned Target.GetFoo() to return type Target. How can that be detected and replaced. And that's an easy example since a method signature is well-defined. What about if type Source is referenced as the operand of and IL operation? Operands are all represented as object. Should the code check whether the operand is a type? What if it's a property of 'Source' that's being referenced?

I wanted to focus on the framework mixins themselves, and this was so much plumbing. So, yeah, I was trying avoid this.[1][2]

I started in November 2013. My family and I were still living in Denmark at the time. I was interrupted in December because my wife and children flew home just before Christmas.

My best-laid January plans were to work on Bix non-stop. That didn't happen. Instead, all of my energy went into getting as much work as possible done during my final days at Statistics Denmark and preparing to move. These things'll wear a guy out. I picked up Bix from time to time, but I mostly split my free time between video calls with my family and watching episodes of Burn Notice.

Throughout all of this time, even when I wasn't working on Bix code cloning, I was thinking about it. Even worn out, 3 episodes into Bruce Campbell-land, it would be sitting at the back of my mind. From time to time, I'd pull out the laptop, attack it, hit a wall, and back off. I would come back later, try a solution, hit a wall, and back off. I'd come later, delete all of my changes, try again, and back off yet again.

I flew out of Copenhagen on the morning of February 1, 2014. After a period of settling in, I was able to put some real time into Bix. Given the pregnancy status of my wife, the period of productive Bix work was limited. On February 28, our new beautiful baby girl disembarked from the mothership.

In the midst of all of this, somehow I got code cloning working. It's not yet feature complete, but it's usable, and the concept has been proven. It supports nested types, it redirects all references to types/properties/fields/etc within the root source type to the target version's of that member. It's useful enough that I decided to spin it off into it's own project.

Bix.Mixers.Fody

The Fody README states:

Manipulating the IL of an assembly as part of a build requires a significant amount of plumbing code. This plumbing code involves knowledge of both the MSBuild and Visual Studio APIs. Fody attempts to eliminate that plumbing code through an extensible add-in model.

I initially wrote the mixing functionality to run as a stand-alone executable with the full knowledge (and a code TODO stating as much) that it would need to change. The ideal solution would be to have apply the mixing (or "code weaving" as post-compile IL manipulation is known) as a build step. Fody provides a framework for exactly this. Perfect :-)

The plan is to pull Bix.Mixers.Fody into its own project, make it available via NuGet, and make Bix include it as a NuGet dependency. I have a "Fodify" feature branch in Bix where I am doing just that. I am currently working directly against the master branch of the Bix.Mixers.Fody project.

Although the mixin functionality is not yet "feature complete" (generic support and event support are notably absent), I plan to make the first official pre-release as soon as the code runs correctly through Fody with fully documented code and a full set of tests. After that, I have about 13 TODOs in the code marking either missing features or research points to examine more closely for correctness, some of which I'll address quickly, and others I'll address as they become roadblocks in the future.

After the mixin command, my next task is planned to be some sort of projection weaver, so that, for example, a rich business object can project properties to create a simple POCO or DTO.

[1] PostSharp politely but firmly declined my request for a free open-source project license or I could have avoided all of this manual code weaving. At the time, the code base was nearly non-existent, so I can't say I blame them. It does illustrate a chicken/egg scenario for this type of project. At this point, I'm really enjoying the project, so I'm content with how it has turned out.

[2] At the suggestion of a colleague, I looked into Fody for a mixin plugin before I started this project. At the time, there was none. There is now a Fody mixins plugin. I've already built my own to perfectly fit my needs, however, so I won't be switching.

Baby Shoes: A Communication Exercise

| Comments

The Request

I was sitting on the couch in the front room yesterday when I recieved the following text out of the blue from my wife.

shoes-thanks.png

I don't remember precisely what I was doing when the message came in, but the odds are good that I was watching an episode of Burn Notice. The point, though, is that I was caught completely off guard with the request.

It took me a minute to reply.

Some Context

We are both excited by the imminent arrival of our first daughter in mid-March. I wasn't sure why my wife was entrusting to me the task of buying her first pair of shoes, but her message made it seem like she was pretty excited about the idea. To be honest, it made me feel special to be specifically included in a task that I normally wouldn't have any confidence in doing "right". This was especially true because I'm feeling so disconnected from them right now: my wife and sons flew back to the US two weeks ago, and I won't be joining them for another 4 weeks.

And, besides, who am I to turn down cool points from my favorite person?

Getting Used to the Idea

I had questions. I should ask, I thought, what colors I should be looking for. I should ask about styles. I should ask whether these would be for the outfit that she'd wear home from the hospital.

Then I realized that it would be counter-productive to just blindly ask questions. I should look at what is available first. I should look more closely at the pictures that I've seen of clothes that my wife has already bought. I could ask more informed questions later. I didn't want to shirk on my newly assigned duty, after all.

My Reply

So I would agree, do my research, ask her opinion later if needed, and then buy some shoes online to have shipped to her.

In texting, "OK" sometimes seems like too a curt reply. It's fine for a request to pick up milk or bread on the way home, but I wanted to communicate that not only was I okay with the request but that I also appreciated being asked; in fact, I wanted to focus primarily on that. So:

shoes-ok.png

Vantage Points (An Aside)

Imagine that you and an enemy with a gun are traveling in a transport plane, and that the enemy, standing at the front of the cargo area, shoots at you where you are standing at the back of the vehicle. From your viewpoint, a bullet just shot into your bullet-proof vest (because you were prepared). It seems pretty clear-cut.

But now imagine that the vehicle is traveling fast, and not just fast, but faster than the bullet can shoot out of the gun. The bullet would come out of the gun moving away from you. Unfortunately, it won't be moving fast enough. A high-speed camera on the ground would catch an image of your body slamming forward into the bullet.[1]

(Mis)communication

In retrospect, my surprise at the request should have tipped me off. Throughout the day, we continued to text back and forth about other random things, like we normally do, and I went to bed none the wiser to what the astute reader may have already guessed: I sent an ambiguous reply to a completely misinterpreted message.

Somewhere about 1:00 AM my time (this would be 6-ish her time), my wife posted a public thank you to me on FaceBook, awarding me 1000 awesome points for having picked out and shipped our daughter's first pair of shoes.

Fallout

Her text woke me at about 1:40 AM. She was genuinely confused. Why had I taken taken credit for buying shoes that I hadn't bought?

I had just learned that the shoes existed, so I couldn't have taken credit on purpose. I felt amused and a little let down. Now it all made since. I told her what I had thought, and she told me what she had meant. I said that I had planned to shop the next day.

Then she told me about the public post for all of our friends and family to read and about the private message from her friend who had actually bought the shoes. My amusement faded.

I felt awful.

Tinderbox

Both of us had, in spite of actions being out-of-character, attributed something positive to the other. Both of us were let down. I had also caused her to publicly make a false statement that took credit away from her generous friend.

Situations like this arise from time to time in relationships. It's like suddenly finding yourself and your partner standing in a wooden room full dry kindling, each holding a lit match. A wrong word, an incorrect attribution of intent, a failure to overcome a bruised ego: any one of these things could spark a firestorm. Every couple has their own dynamic, but whatever the means, both of you must care very much about to extricating yourselves without causing damage.

I'm proud to say that we made it.

And somebody owes his wife flowers.

[1] This half-remembered story comes from a too-long-ago reading of Time: A Traveler's Guide.

Myra, yer mom died

| Comments

Miles away, a raucous gathering of moisture, darkness, and transience played catch with shafts of pure, hot light, each toss accompanied by a mosh pit of billions of molecules of nitrogen, oxygen, and their friends each bumping their neighbors outward to make room for their own individual, short-lived dance of excitement. The sound of this dance in the distance once again brought Floyd back to the present, where the earth continued to turn underneath his still head.

Floyd turned his head ever so slightly and then just a bit further to the right; he saw on the horizon, churning in on itself, never standing still, not quite moving, shades of dancing blackness, visible as an absence of stars and a hint of the movement of a monster hiding in a child's closet. As he watched, an illumination flickered, lighting the storm from behind, making visible one sliver of darkness upon which Floyd's eyes locked as it broke from the mass.

As the lightning faded, the small cloud remained locked in Floyd's vision. An ethereal surfer, it gathered its mass, shifting from vapor to liquid as it rode a wave of thunder. Floyd first heard the vanguard of a sound, a subtle shifting of air scuffing against itself.

And then somebody punched Floyd's eardrums and rattled the rocks in the ground around him, and all of the deer, coyotes, bears, and bison jumped out of their hiding spots, and all of the trout, carp, turtles, and tadpoles shot up out of their watery homes, and all of the crows, jays, robins, and sparrows flew down from their nests, and they pointed and they laughed terrible and loud laughs, and the cloud was still flying toward him like the gift of a sniper, a confused sniper with perfect aim and the cacophony ground its heels into Floyds brain.

Silence. The slight scuff of the sound of Floyd being dragged across the grassy ground, but otherwise nothing.

Straining to focus on what was now a single drop of water, Floyd's eyes, for just a hint of a moment, crossed as the droplet pulled his focus too close before landing in the center of his forehead, a cold, crisp drip.

Floyd's eyes uncrossed as he relaxed his gaze. He noticed a background white noise, static, in the distance, punctuated by cracks of thunder, loud and intermittent. The noise grew louder, and Floyd noticed that the stars did not stretch out as far as before. They were, in fact, disappearing, being tucked in to a great black blanket. And the static was louder. And the static stepped from the background to the foreground. And then a river began flowing directly out of the sky, washing over Floyd.

"Myra, yer mom died," Floyd croaked. "Sumbuddy hitter with uh truck."

Bix Field Initialization Question

| Comments

I've added a field called Mixers into the IMixes interface in Bix. This field needs intialized. No problem, I thought at first, I've definitely seen a property for initial value on fields.

Right?

So I set an initial value in the reference assembly like this. (The attributes are because the code is being generated by a compile-time tool and because the field should not be made available to development tools.)

[CompilerGenerated]
[SpecialName]
private ReadOnlyCollection<IMixer> mixers = new ReadOnlyCollection<IMixer>(new IMixer[0]);

Good so far. Now to compile and view it in ILDasm. Here's the field declaration.

.field private specialname class [mscorlib]System.Collections.ObjectModel.ReadOnlyCollection`1<class [Bix.Mix]Bix.Mix.IMixer> mixers
.custom instance void [mscorlib]System.Runtime.CompilerServices.CompilerGeneratedAttribute::.ctor() ( 01 00 00 00 )

The first line above is the field declaration, and the second is the CompilerGeneratedAttribute. (The SpecialNameAttribute is converted into the specialname keyword on the field definition.) Nowhere in this simple CIL is the field's initial value set.

So where the field initialized?

The answer is that the field is initialized in the constructor. If you are familiar with C#, then you may be under the impression that every constructor calls a base type's constructor as its first step. From the logical standpoint of the C# language, this is true. From the standpoint of CIL, however, it's less true. As it turns out, code to initialize each instance field runs directly within each constructor, and it runs before the base constructor is called.

There are currently two constructors in the reference assembly's version of EncapsulatesTestType: one parameterless and one for deserialization. The C# compiler generates the following code at the beginning of each.

IL_0000:  ldarg.0
IL_0001:  ldc.i4.0
IL_0002:  newarr     [Bix.Mix]Bix.Mix.IMixer
IL_0007:  newobj     instance void class [mscorlib]System.Collections.ObjectModel.ReadOnlyCollection`1<class [Bix.Mix]Bix.Mix.IMixer>::.ctor(class [mscorlib]System.Collections.Generic.IList`1<!0>)
IL_000c:  stfld      class [mscorlib]System.Collections.ObjectModel.ReadOnlyCollection`1<class [Bix.Mix]Bix.Mix.IMixer> Bix.Mixers.MixerTestTargets.EncapsulatesTestType::mixers
IL_0011:  ldarg.0
IL_0012:  call       instance void [mscorlib]System.Object::.ctor()

This code initializes the field to the dummy value which the reference assembly currently specifies, and then it calls the base constructor. If you think about it, it's sort of an AOP strategy to handle C#-like field initialization in CIL.

So now I have to worry about how to initialize the fields and keep them testable. I can easily push the code into all constructors, but I will need to worry about order. I could then sort the field initilizations in a deterministic way and compare. Unfortunately, that leaves the problem of separating generated fields from non-generated. Once that's solved, I will have to go a step further and separate fields generated by different behavior mixers.

I think that's getting a little too complex. I believe that I have an alternate solution in the works, but I will hold off on writing about it for now.

A Boy Named Sy

| Comments

You need to know about A Boy Named Sue, a song written by Shel Silverstein and made famous by the man in black himself.

Fun story, right? But who would name a boy Sue for real? Well, funny story...

I was talking to a Danish couple who came into the Scanningsklinikkens while Rachelle was in the sonogram room learning that we're having a girl. While she was getting this wonderful news, I was learning about Danish pronounciations of Sy's name. The more standard spelling of Sy's name, "Silas" is quite common over here, and it is pronounced like "See-las", while our spelling, "Sylas", would be prounced here in Denmark like "Soo-las"...so "Sy" would become "Sue".

I hope he never comes seeking revenge.

Reference Assembly for Bix Mixers

| Comments

In building the Bix CecilMixer, which uses Mono.Cecil to perform the post-compile mixing (a.k.a. code weaving) step, I've found myself making heavy use of ildasm to examine compiled assemblies. Up until now, I've mostly been creating code similar to what I want on an ad-hoc basis, compiling it, and then disassembling it. I'm still in very early development, and this is already starting to become a lot of overhead.

It's time to formalize and improve this strategy.

I've already created a test mixer target assembly that I'm using to test mixing functionality, so I have a well-known starting point for mixing. I also know how a C# version of the mixed code should look (after all, I've been creating small, ad-hoc pieces in C# and compiling as needed); the next logical step is to create and maintain a reference version of the post-mixed assembly. This will allow me to "save" my work, so to speak, and it will allow the reference assembly to become part of the unit testing process. I might even drop other unit tests in favor of a single deep assembly comparison test. (I haven't made that decision yet, but don't be surprised if you see it happen.)

So far, this is natural evolutionary step devoid of any hard decisions. It does present a task in performing the deep structure comparison that is arduous, if not necessarily difficult. It's a lot of up-front work, and I see the possibility of needing to tinker with it a lot as the code base evolves. The basic idea is straightforward, though, in that the two assemblies have different filenames but should contain the same namespaces, type names, and everything else. After mixing the test target, a deep structure comparison should show only a difference in module names (because module names correspond to filenames). To reference both projects from the test project, I simply have to give one of the references, the reference assembly in this case, an alias. (Sorry for causing parsing difficulties with the two different uses of "reference" in that last sentence.)

Now: should I test the generated IL code along with the structure?

It seems like I won't know the "right" answer to this for some time. Because I will generate the code based on disassembly of the compiled referenced code (minus extra nop instructions), all I will need to do for initial, simplistic testing is to compare instructions with nop ignored. Up front, I plan to do this, but I am certain that this will too easily break. For example, compiling in Release mode might cause enough difference to break the code. Using the mono compiler might break it. Upgrading the .NET SDK might break it.

For now, I'll be taking the pragmatic direction of writing IL comparison tests that are usable for me during my current development but that will be easy to comment out/remove/put #ifdef around if needed. I assume that I will eventually need to put time into solving this more generally, but I'll not be shaving that yak today.

Exiting the Bus

| Comments

Last week on my way to work, I stood at the front of the venerable 4A bus waiting and balancing as the driver slowed for the St Kjelds Plads stop. I had adopted a leisure stance, and the obviously in-a-rush patron behind me was cramping my style. As the bus lurched to its final stop, inertia pushed me forward, and I went with it, deftly segueing from pure leisure to pure efficiency without effort, bursting toward the door that opened for me as if on cue, leaving Ms. annoying-hurry-GETOFFMYBACKALREADY in the metaphorical dust. I was validated! I was venerated! See, lady, this is how you do bus.

Man that felt good.

Every morning since then, I've tried to recreate that moment. Every morning since I've stumbled and cursed and nearly run into the bus door.

Introducing Bix: The Business Mix

| Comments

I've got a new project called Bix that I'm working on during what free time I can carve out. The idea is that rather than inheriting all of the infrastructure for a framework from base type, you can use .NET attributes to pick which framework behaviors you want mixed in with your class. These behaviors are then mixed in using post-compilation code weaving with mono.cecil, which is an aspect oriented programming (AOP) technique. This frees you up to focus solely on domain modeling.

It also allows you to pick and choose the behaviors that you want to use per-type (or per base-type). And I really hope to provide good support for Domain Driven Design by making the domain model the core of the application. Frameworks that I've used often feel like wrapping behavior around a core of relational database tables. Ugh.

Then there's this: a pet peeve of mine is mixing infrastructure inheritance and domain inheritance. These are not standard terms, so I should explain.

Say you're using CSLA (which is a fine framework, btw). You would inherit a domain type from the base business object type. This is an infrastructure inheritance in that your Foo is-a business object. That's a perfectly legitimate relationship. That the base business object has no meaning within the domain is not consequential because you're interested in grabbing shared behaviors.

Now say that you have a domain type Bar that, in the domain model, is-a Foo. So you implement that relationship with inheritance. This is also legitimate. Because Bar and Foo are both meaningful within the domain, this is domain inheritance.

But wait, Bar also is-a business object directly in the same way that Foo is-a business object, so why is one the child of business object while the other is a grandchild.

I understand that Bar still gets all of the business object behavior, and that there is probably nothing wrong here. But it still bothers me. I feel like type inheritance is being used for two very different things in the same hierarchy. One use is plumbing and one use is conceptual within the domain. Having worked on an in-house framework for several months in the past, I've also run into some of the more pragmatic issues that this mixture of inheritance types creates, but though dealing with them added some complexity, none of them were blocking.

Still: it bothers me. So I'm happy to be trying my hand at creating a Better Way.

You can see what exists of my plans on the Bix Trello board.

In the life

| Comments

Today I watched The Godfather part 2 on Bravo. With the commercials and everything.

Today my youngest niece turned one.

Today I ate twice. Spaghetti with garlic bread the first time I got hungry. Ben & Jerry's Half-Baked the second time. I'm hungry again, but for food...?

Today I watched part of Pulp Fiction, also on Bravo. Commercials, etc.

Today I doodled on a canvas drawing pad. Outlined random objects on my desk. Followed up with uninspired connecting lines. A couple of conversation bubbles.

Today I watched Flipped Out on Bravo. Before Pulp Fiction. After The Godfather part 2. The dude (term used loosely) had a psychic bless a house he was selling.

What a great idea.

Today I thought about the girl who must have known she was unnatractive who approached me and Dan at Harvey's last night. She distracted from her looks using bad conversation and worse make-up. Our one-word answers used up her conversation quickly.

Today I lacked motivation to surf channels.

Today I had the great idea to hand my next paycheck to the first psychic I see. I assume that, knowing what I am about to do, one will introduce himself to me on payday.

Today I watched Top Chef on Bravo. Played solitaire on my phone during the commercials.

Tomorrow, I think, I will go for a walk.

Breakin' the law

| Comments

My 8-year-old niece asked me last week why I don't have a girlfriend. "I don't know," I said, "I just don't." She seemed to let it drop after that, and I continued to talk to my sister.

My niece, however, was not content to let it end with that. She walked away for a couple of minutes and then returned to deposit a slip of paper on the kitchen table in front of me.

She wrote me a ticket for $1,000. The crime: "Not Having a Girlfriend".

I played along pretending to be mad for a couple of minutes and then continued once again talking to my sister.

It was a little warm in the house, and I began to fan myself absentmindedly with the "ticket". Apparently this behavior fell outside of the bounds of proper respect for the ticket because I soon found myself holding a second ticket, this one for $2,000. This crime: "Using the Other One as a Fan".

Later she was kind enough to draw me a $3,000-bill, providing me with the funds to pay my fines.