Statistics & Lies

Light on Statistics / Short on Lies

Bix.Mixers 0.1.3 Released

| Comments

Bix.Mixers v0.1.3, a Fody addin for writing custom mixins in C#, is now available via NuGet and direct source and binary download.[1] If you're a C# developer, please test it and provide feedback :-)

This release fixes a bug an issue where implementing a mixin interface member explicitly would cause the code weaver to fail unless a wordaround was implemented.

[1] As indicated by its < 1.0 version number, this is still pre-release software. I have tested it as well as I can, but I make no guarantees that everything works perfectly. Breaking changes could also occur between minor versions. (I'll try not to do that, but it's better to do it sooner than later.) Use it in production at your own risk. However you use it, though, I'm interested in hearing about it!

Bix.Mixers 0.1.2 Released

| Comments

Bix.Mixers v0.1.2, a Fody addin for writing custom mixins in C#, is now available via NuGet and direct source and binary download.[1] If you're a C# developer, please test it and provide feedback :-)

This release fixes a bug discovered by RoboPhred where the weaving process was removing attributes on unrelated members of the target assembly. This was a pretty serious bug, and I'm very thankful for RoboPhred's help in tracking down the issue.

[1] As indicated by its < 1.0 version number, this is still pre-release software. I have tested it as well as I can, but I make no guarantees that everything works perfectly. Breaking changes could also occur between minor versions. (I'll try not to do that, but it's better to do it sooner than later.) Use it in production at your own risk. However you use it, though, I'm interested in hearing about it!

Developer's Guide to Communicating Technical Value

| Comments

Why does a developer need to communicate his or her value to a client?

When I returned to my undergrad alma mater a few years back to speak to the local ACM chapter about software engineering in the real world, the subject matter was not technical. I spoke to them about the value that they must add to their company. If they expect to step out of college into a $50k+ salary, then their work needs to add more than that to the company bottom line each and every year.

This declaration was daunting. College of the Ozarks allows students from lower-income backgrounds to work their way to a Bachelor’s degree. Most who attend this school aren’t accustomed to regularly dealing with even 4 figure sums. Asserting that an employer would expect them to be worth 5+ figures per year got their attention. I went on from there to talk about using soft skills to communicate that value without being a self-important blowhard.

I feel like it's worth delving deeper into this idea, but first I'd like to assert that enabling those on the other side of a company’s technical divide to implement business ideas builds the expected value really quite easily. If this wasn't the case, then developers would be out of jobs.

It’s not enough for the career minded developer to add this value silently; however. A person who has come up with an idea that was implemented by their IT organization will feel that she is responsible for the end result. She is correct. An implementing developer, even a lead or principal, is similar to a plumber or a painter: a skilled worker; maybe not easily replaceable, but not core to the project.

How does a skilled worker communicate his or her value to a client?

Reality for many developers in the corporate world is that requirements land on their desk after being filtered through JADs, business analysts, and project managers. The project will have already been sized and scheduled. The developer becomes a paleontologist, combing through the fossil record of the document to determine original intent.

In such a world, it is difficult for a developer to communicate her own value. Opening technical discussions with the business sponsor of a project, who will almost certainly speak from a viewpoint foreign to the developer, can cause unexpected consequences such as requirements changes and scope creep leading to project delays. This spurs some organizations go so far as to discourage or even disallow developers to interact directly with the business sponsors.

Even outside of this difficulty, trying to have a technical conversation with a business sponsor is often not the best way to communicate value. Imagine a painting contractor hired to paint your office trying to engage you about how the idea to use a paint sprayer in the entry hall was discarded because of the possibilty of paint getting into office workers clothing as they walked by. You'd probably agree with the decision, but you'd also probably think that should seem obvious for a professional painter and you'd wonder why you were being told. What if next time you bumped into this fellow, he tried to explain to you why they chose to use a matte paint? After a few uninteresting exchanges like this, might you tend to avoid that particular worker when you have a choice?[1]

If a toilet flushes, if a painted wall is without visible defects, if a customer pushes a button and the business collects $200 without passing Go, what does the client care whether about, for example, code littered with obtuse variable names? There are myriad discussions around this question in the long term. In the short term, outside of cost, scope, and schedule, I believe that it comes down to whether the client feels validated by her interactions with the skilled worker.

How does a technically minded developer go about providing validation?

Providing validation is an emotional rather than an intellectual exchange. It's not always natural for a technically minded individual to engage in an emotional exchange, and being on the giving end of such an exchange is even more difficult.

Here are some things that have worked for me.

  • I engage the business sponsor about the project without adding anything technical to the conversation. My instinct is to tell them all about the clever way that I’m solving their problem, but what I do is listen to how their solution solves the company’s problem or meets the customers’ needs. The result is that the business sponsor understands that I care about the project. In doing this I took a small step toward communicating my value to the company. The bonus is that there is nearly a 100% chance of learning useful about the project.

  • If a business sponsor engages me a technical discussion, I try to determine the true level of interest before jumping into the details of my newfangled tri-sub-optimizing architectural marvel wonder. It could be that she has a technical background and truly wants to hear about it in all its beautiful and gory detail. Or, maybe she is a natural learner, and she simply wants to expand the borders of her knowledge a bit. It could also be that she herself is looking to communicate her value, and she is looking for me to provide a problem for her to provide input on. I let her guide the conversation.

  • I try to find ways to let the business sponsor provide more direct input. The ultimate validation for a business sponsor is to see her own ideas directly mapped into a system. Domain driven design is a great enabler in this respect. Where company culture does not provide opportunity to collaborate on a ubiquitous language, every little bit helps. For example, if a requirements document calls one concept a Flibber but you think of it as a Flooper, then change your terminology to match the requirements. Call it a Flibber in your code, from database columns to class names to variable names. When you speak about how the Flibber works, the business sponsor who provided the term will understand that her technical team listens to her.

Is this dishonest?

The key for me is in recognizing emotional exchanges for what they are. If you realize that a discussion is not an in-depth technical conversation, then you won't feel so strongly the need to provide the "actually"'s and "no"'s and other interjections that others may find abrasive. The right way to have the convsersation is to find a way to provide validation.

Even so, you must stick to your principles. An exchange like this must be earnest and honest. You have to believe what you say. Those business folk that you're interacting with will likely have a social intelligence that is much more developed than your typical techie's. If you're saying things that you don't believe, there's a good chance they'll know.

The key to these exhanges is to be nice, which means that you need to be nice.

Good luck :-)

[1] Please understand that the painting conversation examples are complete BS. If you are a painter yourself, then you will already know this.

Bix.Mixers 0.1.1 Released

| Comments

Bix.Mixers v0.1.1, a Fody addin for writing custom mixins in C#, is now available via NuGet and direct source and binary download.[1] If you're a C# developer, please test it and provide feedback :-)

[1] As indicated by its < 1.0 version number, this is still pre-release software. I have tested it as well as I can, but I make no guarantees that everything works perfectly. Breaking changes could also occur between minor versions. (I'll try not to do that, but it's better to do it sooner than later.) Use it in production at your own risk. However you use it, though, I'm interested in hearing about it!

Bix.Mixers.Fody: Approaching Release

| Comments

The Bix spinoff project that provides C# mixins is nearing release. I have very good test coverage with no known bugs.

[Updated 2014-04-20 to remove generics support from the initial release]

What's Done?

  • Mixin target types are marked with an attribute that specifies the mixin definition interface.
  • Mixin implementation types are specified in configuration. One implementation type is configured per definition interface.
  • The mixin definition interface gets added to the mixin target type.
  • Fields, methods, properties, events, and nested types are copied from the non-abstract mixin implementation type to the target type.
  • Static members of mixin implementation types are copied into the target type, as well as instance members.
  • Member are copied from the mixin implementation type to the mixin target type by default. It does not matter whether they are part of the mixin definition interface.
  • Annotating a mixin implementation member with a SkipAttribute will keep the mixer from copying that member to the target type.
  • Nested reference types, values types, interfaces, enumerations, and delegates are all supported.
  • Nested types may be abstract.
  • Implementation type methods may be virtual. Methods within nested types may be virual or abstract.
  • Interfaces and events can be implemented implicitly or explicitly in the mixin, and they will be mixed into the target type as they are written.

What's Left?

  • Support for generic methods is on track for this release.
  • Support for nested open generic types is also on track for this release.
  • Mixin reference redirection is complete, but it is only partially tested. It will be fully tested for this release. [1]
  • A decision on what to do about type initializers, a.k.a. C# static constructors, needs to be made.
  • Currently constructors are copied along with everything else, which means that you have to be careful about putting SkipAttributes on empty constructors. Constructor control should be more intelligent, and this release may or may not change constructor handling.
  • Several unsupported scenarios need to be explicitly checked for so that errors and warnings can be raised.
  • Some work needs done on simplifying configuration.
  • Documentation, both code level and project level, needs completed.
  • The NuGet package needs to be created and tested.

What's Unsupported

  • Field initial values will not be supported in this release. In IL, field initialization happens in constructors before calling the base constructor, so any movement on this is dependent on final decisions on constructor behavior.
  • Mixin definition interfaces are not allowed to be generic.
  • Mixin implementation types are not allowed to be generic.
  • Mixin implementation types are not allowed to be abstract.
  • Mixin implementation types are not allowed to inherit from any type other than System.Object
  • Mixin implementation types are not allowed to implement interfaces other than the Mixin definition interface.
  • Naming collisions are not resolved. Member name collisions between a mixin implementation and a target will cause an error. For members of the mixin definition interface, this can be avoided by implementing members explicitly. For members that are not defined on the interface, there is no remediation.
  • Support for generic methods and nested open generic types will not be included in this release [Added 2014-04-20]

When?

If this is not completed within the next week, then it will likely another month after that, as my family and I are moving in two weeks, and that will soon start to dominate my time. I will do everything I can to finish in the next week, which would be by 4/25.

[1] Mixin reference redirection occurs when an implementation member references another member. Upon mixing, that reference is modified to point to the version of the member within the mix target.

Bix Updates: Mixin via Cloning and Fody

| Comments

Bix now supports C# interface-based mixins on the master branch. I am currently working on a feature branch where the mixer functionality has been spun off into a new project called Bix.Mixers.Fody.

That's pretty much what I have to say. Feel free to read on for the long version :-)

Bix Mixin Functionality

At last checkin, my concern with was about field initialization and how to handle constructors. In looking to solve that problem, I couldn't figure out good way to make field initialization (and constructor logic, in general) controllable within the strategy of decompiling a reference assembly and manually creating matching IL.

So I pulled the trigger on a big change, a change that I had been avoiding because of the daunting amount of IL level work and because I just wasn't certain that it would work. There were hard problems to solve. For example, if I'm cloning a method Source.GetFoo(), and its return value is of type Source, then the intent is likely for the cloned Target.GetFoo() to return type Target. How can that be detected and replaced. And that's an easy example since a method signature is well-defined. What about if type Source is referenced as the operand of and IL operation? Operands are all represented as object. Should the code check whether the operand is a type? What if it's a property of 'Source' that's being referenced?

I wanted to focus on the framework mixins themselves, and this was so much plumbing. So, yeah, I was trying avoid this.[1][2]

I started in November 2013. My family and I were still living in Denmark at the time. I was interrupted in December because my wife and children flew home just before Christmas.

My best-laid January plans were to work on Bix non-stop. That didn't happen. Instead, all of my energy went into getting as much work as possible done during my final days at Statistics Denmark and preparing to move. These things'll wear a guy out. I picked up Bix from time to time, but I mostly split my free time between video calls with my family and watching episodes of Burn Notice.

Throughout all of this time, even when I wasn't working on Bix code cloning, I was thinking about it. Even worn out, 3 episodes into Bruce Campbell-land, it would be sitting at the back of my mind. From time to time, I'd pull out the laptop, attack it, hit a wall, and back off. I would come back later, try a solution, hit a wall, and back off. I'd come later, delete all of my changes, try again, and back off yet again.

I flew out of Copenhagen on the morning of February 1, 2014. After a period of settling in, I was able to put some real time into Bix. Given the pregnancy status of my wife, the period of productive Bix work was limited. On February 28, our new beautiful baby girl disembarked from the mothership.

In the midst of all of this, somehow I got code cloning working. It's not yet feature complete, but it's usable, and the concept has been proven. It supports nested types, it redirects all references to types/properties/fields/etc within the root source type to the target version's of that member. It's useful enough that I decided to spin it off into it's own project.

Bix.Mixers.Fody

The Fody README states:

Manipulating the IL of an assembly as part of a build requires a significant amount of plumbing code. This plumbing code involves knowledge of both the MSBuild and Visual Studio APIs. Fody attempts to eliminate that plumbing code through an extensible add-in model.

I initially wrote the mixing functionality to run as a stand-alone executable with the full knowledge (and a code TODO stating as much) that it would need to change. The ideal solution would be to have apply the mixing (or "code weaving" as post-compile IL manipulation is known) as a build step. Fody provides a framework for exactly this. Perfect :-)

The plan is to pull Bix.Mixers.Fody into its own project, make it available via NuGet, and make Bix include it as a NuGet dependency. I have a "Fodify" feature branch in Bix where I am doing just that. I am currently working directly against the master branch of the Bix.Mixers.Fody project.

Although the mixin functionality is not yet "feature complete" (generic support and event support are notably absent), I plan to make the first official pre-release as soon as the code runs correctly through Fody with fully documented code and a full set of tests. After that, I have about 13 TODOs in the code marking either missing features or research points to examine more closely for correctness, some of which I'll address quickly, and others I'll address as they become roadblocks in the future.

After the mixin command, my next task is planned to be some sort of projection weaver, so that, for example, a rich business object can project properties to create a simple POCO or DTO.

[1] PostSharp politely but firmly declined my request for a free open-source project license or I could have avoided all of this manual code weaving. At the time, the code base was nearly non-existent, so I can't say I blame them. It does illustrate a chicken/egg scenario for this type of project. At this point, I'm really enjoying the project, so I'm content with how it has turned out.

[2] At the suggestion of a colleague, I looked into Fody for a mixin plugin before I started this project. At the time, there was none. There is now a Fody mixins plugin. I've already built my own to perfectly fit my needs, however, so I won't be switching.

Baby Shoes: A Communication Exercise

| Comments

The Request

I was sitting on the couch in the front room yesterday when I recieved the following text out of the blue from my wife.

shoes-thanks.png

I don't remember precisely what I was doing when the message came in, but the odds are good that I was watching an episode of Burn Notice. The point, though, is that I was caught completely off guard with the request.

It took me a minute to reply.

Some Context

We are both excited by the imminent arrival of our first daughter in mid-March. I wasn't sure why my wife was entrusting to me the task of buying her first pair of shoes, but her message made it seem like she was pretty excited about the idea. To be honest, it made me feel special to be specifically included in a task that I normally wouldn't have any confidence in doing "right". This was especially true because I'm feeling so disconnected from them right now: my wife and sons flew back to the US two weeks ago, and I won't be joining them for another 4 weeks.

And, besides, who am I to turn down cool points from my favorite person?

Getting Used to the Idea

I had questions. I should ask, I thought, what colors I should be looking for. I should ask about styles. I should ask whether these would be for the outfit that she'd wear home from the hospital.

Then I realized that it would be counter-productive to just blindly ask questions. I should look at what is available first. I should look more closely at the pictures that I've seen of clothes that my wife has already bought. I could ask more informed questions later. I didn't want to shirk on my newly assigned duty, after all.

My Reply

So I would agree, do my research, ask her opinion later if needed, and then buy some shoes online to have shipped to her.

In texting, "OK" sometimes seems like too a curt reply. It's fine for a request to pick up milk or bread on the way home, but I wanted to communicate that not only was I okay with the request but that I also appreciated being asked; in fact, I wanted to focus primarily on that. So:

shoes-ok.png

Vantage Points (An Aside)

Imagine that you and an enemy with a gun are traveling in a transport plane, and that the enemy, standing at the front of the cargo area, shoots at you where you are standing at the back of the vehicle. From your viewpoint, a bullet just shot into your bullet-proof vest (because you were prepared). It seems pretty clear-cut.

But now imagine that the vehicle is traveling fast, and not just fast, but faster than the bullet can shoot out of the gun. The bullet would come out of the gun moving away from you. Unfortunately, it won't be moving fast enough. A high-speed camera on the ground would catch an image of your body slamming forward into the bullet.[1]

(Mis)communication

In retrospect, my surprise at the request should have tipped me off. Throughout the day, we continued to text back and forth about other random things, like we normally do, and I went to bed none the wiser to what the astute reader may have already guessed: I sent an ambiguous reply to a completely misinterpreted message.

Somewhere about 1:00 AM my time (this would be 6-ish her time), my wife posted a public thank you to me on FaceBook, awarding me 1000 awesome points for having picked out and shipped our daughter's first pair of shoes.

Fallout

Her text woke me at about 1:40 AM. She was genuinely confused. Why had I taken taken credit for buying shoes that I hadn't bought?

I had just learned that the shoes existed, so I couldn't have taken credit on purpose. I felt amused and a little let down. Now it all made since. I told her what I had thought, and she told me what she had meant. I said that I had planned to shop the next day.

Then she told me about the public post for all of our friends and family to read and about the private message from her friend who had actually bought the shoes. My amusement faded.

I felt awful.

Tinderbox

Both of us had, in spite of actions being out-of-character, attributed something positive to the other. Both of us were let down. I had also caused her to publicly make a false statement that took credit away from her generous friend.

Situations like this arise from time to time in relationships. It's like suddenly finding yourself and your partner standing in a wooden room full dry kindling, each holding a lit match. A wrong word, an incorrect attribution of intent, a failure to overcome a bruised ego: any one of these things could spark a firestorm. Every couple has their own dynamic, but whatever the means, both of you must care very much about to extricating yourselves without causing damage.

I'm proud to say that we made it.

And somebody owes his wife flowers.

[1] This half-remembered story comes from a too-long-ago reading of Time: A Traveler's Guide.

Myra, yer mom died

| Comments

Miles away, a raucous gathering of moisture, darkness, and transience played catch with shafts of pure, hot light, each toss accompanied by a mosh pit of billions of molecules of nitrogen, oxygen, and their friends each bumping their neighbors outward to make room for their own individual, short-lived dance of excitement. The sound of this dance in the distance once again brought Floyd back to the present, where the earth continued to turn underneath his still head.

Floyd turned his head ever so slightly and then just a bit further to the right; he saw on the horizon, churning in on itself, never standing still, not quite moving, shades of dancing blackness, visible as an absence of stars and a hint of the movement of a monster hiding in a child's closet. As he watched, an illumination flickered, lighting the storm from behind, making visible one sliver of darkness upon which Floyd's eyes locked as it broke from the mass.

As the lightning faded, the small cloud remained locked in Floyd's vision. An ethereal surfer, it gathered its mass, shifting from vapor to liquid as it rode a wave of thunder. Floyd first heard the vanguard of a sound, a subtle shifting of air scuffing against itself.

And then somebody punched Floyd's eardrums and rattled the rocks in the ground around him, and all of the deer, coyotes, bears, and bison jumped out of their hiding spots, and all of the trout, carp, turtles, and tadpoles shot up out of their watery homes, and all of the crows, jays, robins, and sparrows flew down from their nests, and they pointed and they laughed terrible and loud laughs, and the cloud was still flying toward him like the gift of a sniper, a confused sniper with perfect aim and the cacophony ground its heels into Floyds brain.

Silence. The slight scuff of the sound of Floyd being dragged across the grassy ground, but otherwise nothing.

Straining to focus on what was now a single drop of water, Floyd's eyes, for just a hint of a moment, crossed as the droplet pulled his focus too close before landing in the center of his forehead, a cold, crisp drip.

Floyd's eyes uncrossed as he relaxed his gaze. He noticed a background white noise, static, in the distance, punctuated by cracks of thunder, loud and intermittent. The noise grew louder, and Floyd noticed that the stars did not stretch out as far as before. They were, in fact, disappearing, being tucked in to a great black blanket. And the static was louder. And the static stepped from the background to the foreground. And then a river began flowing directly out of the sky, washing over Floyd.

"Myra, yer mom died," Floyd croaked. "Sumbuddy hitter with uh truck."

Bix Field Initialization Question

| Comments

I've added a field called Mixers into the IMixes interface in Bix. This field needs intialized. No problem, I thought at first, I've definitely seen a property for initial value on fields.

Right?

So I set an initial value in the reference assembly like this. (The attributes are because the code is being generated by a compile-time tool and because the field should not be made available to development tools.)

[CompilerGenerated]
[SpecialName]
private ReadOnlyCollection<IMixer> mixers = new ReadOnlyCollection<IMixer>(new IMixer[0]);

Good so far. Now to compile and view it in ILDasm. Here's the field declaration.

.field private specialname class [mscorlib]System.Collections.ObjectModel.ReadOnlyCollection`1<class [Bix.Mix]Bix.Mix.IMixer> mixers
.custom instance void [mscorlib]System.Runtime.CompilerServices.CompilerGeneratedAttribute::.ctor() ( 01 00 00 00 )

The first line above is the field declaration, and the second is the CompilerGeneratedAttribute. (The SpecialNameAttribute is converted into the specialname keyword on the field definition.) Nowhere in this simple CIL is the field's initial value set.

So where the field initialized?

The answer is that the field is initialized in the constructor. If you are familiar with C#, then you may be under the impression that every constructor calls a base type's constructor as its first step. From the logical standpoint of the C# language, this is true. From the standpoint of CIL, however, it's less true. As it turns out, code to initialize each instance field runs directly within each constructor, and it runs before the base constructor is called.

There are currently two constructors in the reference assembly's version of EncapsulatesTestType: one parameterless and one for deserialization. The C# compiler generates the following code at the beginning of each.

IL_0000:  ldarg.0
IL_0001:  ldc.i4.0
IL_0002:  newarr     [Bix.Mix]Bix.Mix.IMixer
IL_0007:  newobj     instance void class [mscorlib]System.Collections.ObjectModel.ReadOnlyCollection`1<class [Bix.Mix]Bix.Mix.IMixer>::.ctor(class [mscorlib]System.Collections.Generic.IList`1<!0>)
IL_000c:  stfld      class [mscorlib]System.Collections.ObjectModel.ReadOnlyCollection`1<class [Bix.Mix]Bix.Mix.IMixer> Bix.Mixers.MixerTestTargets.EncapsulatesTestType::mixers
IL_0011:  ldarg.0
IL_0012:  call       instance void [mscorlib]System.Object::.ctor()

This code initializes the field to the dummy value which the reference assembly currently specifies, and then it calls the base constructor. If you think about it, it's sort of an AOP strategy to handle C#-like field initialization in CIL.

So now I have to worry about how to initialize the fields and keep them testable. I can easily push the code into all constructors, but I will need to worry about order. I could then sort the field initilizations in a deterministic way and compare. Unfortunately, that leaves the problem of separating generated fields from non-generated. Once that's solved, I will have to go a step further and separate fields generated by different behavior mixers.

I think that's getting a little too complex. I believe that I have an alternate solution in the works, but I will hold off on writing about it for now.

A Boy Named Sy

| Comments

You need to know about A Boy Named Sue, a song written by Shel Silverstein and made famous by the man in black himself.

Fun story, right? But who would name a boy Sue for real? Well, funny story...

I was talking to a Danish couple who came into the Scanningsklinikkens while Rachelle was in the sonogram room learning that we're having a girl. While she was getting this wonderful news, I was learning about Danish pronounciations of Sy's name. The more standard spelling of Sy's name, "Silas" is quite common over here, and it is pronounced like "See-las", while our spelling, "Sylas", would be prounced here in Denmark like "Soo-las"...so "Sy" would become "Sue".

I hope he never comes seeking revenge.