Resolving PowerShell Module Assembly Dependency Conflicts

When writing a PowerShell module, especially a binary module (i.e. one written in a language like C# and loaded into PowerShell as an assembly/DLL), it’s natural to take dependencies on other packages or libraries to provide functionality.

Taking dependencies on other libraries is usually desirable for code reuse. However, PowerShell always loads assemblies into the same context, and this can present issues when a module’s dependencies conflict with already-loaded DLLs, preventing two otherwise unrelated modules from being used together in the same PowerShell session. If you’ve been hit by this yourself, you would have seen an error message like this:

In this blog post, we’ll look at some of the ways dependency conflicts can arise in PowerShell, and some of the ways to mitigate dependency conflict issues. Even if you’re not a module author, there are some tricks in here that might help you with dependency conflicts arising in modules that you use.

Contents

This is a fairly long blog post, so here’s a table of contents:

• Why do dependency conflicts occur?
• Quick fixes and their limitations
• More robust solutions
• Solutions for dependency conflicts that don’t work for PowerShell
• Solving the issue in PowerShell itself…

Why do dependency conflicts occur?

In .NET, dependency conflicts arise when two versions of the same assembly are loaded into the same Assembly Load Context (this term means slightly different things on different .NET platforms, which we’ll cover later). This conflict issue is a common problem that occurs essentially everywhere in software where versioned dependencies are used. Because there are no simple solutions (and because many dependency resolution frameworks try to solve the problem in different ways), this situation is often called “dependency hell”.

Conflict issues are compounded by the fact that a project almost never deliberately or directly depends on two versions of the same dependency, but instead depends on two different dependencies that each require a different version of the same dependency.

For example, say your .NET application, DuckBuilder , brings in two dependencies, to perform parts of its functionality and looks like this:

Because Contoso.ZipTools and Fabrikam.FileHelpers both depend on Newtonsoft.Json , but different versions, there may be a dependency conflict, depending on how each dependency is loaded.

Conflicting with PowerShell’s dependencies

In PowerShell, the dependency conflict issue is exacerbated because PowerShell modules, and PowerShell’s own dependencies, are all loaded into the same shared context. This means the PowerShell engine and all loaded PowerShell modules must not have conflicting dependencies.

One scenario in which this can cause issues is where a module has a dependency that conflicts with one of PowerShell’s own dependencies. A classic example of this is Newtonsoft.Json:

In this example, the module FictionalTools depends on Newtonsoft.Json version 12.0.3 , which is a newer version of Newtonsoft.Json than 11.0.2 that ships in the example PowerShell. (To be clear, this is an example and PowerShell 7 currently ships with Newtonsoft.Json 12.0.3.)

Because the module depends on a newer version of the assembly, it won’t accept the version that PowerShell already has loaded, but because PowerShell has already loaded a version of the assembly, the module can’t load its own using the conventional load mechanism.

Conflicting with another module’s dependencies

Another common scenario in PowerShell is that a module is loaded that depends on one version of an assembly, and then another module is loaded later that depends on a different version of that assembly.

This often looks like the following:

In the above case, the FictionalTools module requires a newer version of Microsoft.Extensions.Logging than the FilesystemManager module.

Let’s imagine these modules load their dependencies by placing the dependency assemblies in the same directory as the root module assembly and allowing .NET to implicitly load them by name. If we are running PowerShell 7 (on top of .NET Core 3.1), we can load and run FictionalTools and then load and run FilesystemManager without issue. However, if in a new session we load and run FilesystemManager and then FictionalTools , we will encounter a FileLoadException from the FictionalTools command, because it requires a newer version of Microsoft.Extensions.Logging than the one loaded, but cannot load it because an assembly of the same name has already been loaded.

PowerShell and .NET

PowerShell runs on the .NET platform, and since we’re discussing assembly dependency conflicts, we must discuss .NET. .NET is ultimately responsible for resolving and loading assembly dependencies, so we must understand how .NET operates here to understand dependency conflicts.

We must also confront the fact that different versions of PowerShell run on different .NET implementations, which respectively have their own mechanisms for assembly resolution.

In general, PowerShell 5.1 and below run on .NET Framework, while PowerShell 6 and above run on .NET Core. These two flavours of .NET load and handle assemblies somewhat differently, meaning resolving dependency conflicts can vary depending on the underlying .NET platform.

Assembly Load Contexts

In this post, we’ll use the term “Assembly Load Context” (ALC) frequently. An Assembly Load Context is a .NET concept that essentially means a runtime namespace into which assemblies are loaded and within which assemblies’ names are unique. This concept allows assemblies to be uniquely resolved by name in each ALC.

Assembly reference loading in .NET

The semantics of assembly loading (whether versions clash, whether that assembly’s dependencies are handled, etc.) depend on both the .NET implementation (.NET Core vs .NET Framework) and the API or .NET mechanism used to load a particular assembly.

Rather than go into deep detail describing these here, there is a list of links in thesection that go into great detail on how .NET assembly loading works in each .NET implementation.

In this blog post, we’ll refer to the following mechanisms:

Assembly.Load(AssemblyName)
Assembly.LoadFrom()
Assembly.LoadFile()

The way these APIs work has changed in subtle ways between .NET Core and .NET Framework, so it’s worth reading through the further reading links.

Differences in .NET Framework vs .NET Core

Importantly, Assembly Load Contexts and other assembly resolution mechanisms have changed between .NET Framework and .NET Core.

In particular .NET Framework has the following features:

• The Global Assembly Cache, for machine-wide assembly resolution
• Application Domains, which work like in-process sandboxes for assembly isolation, but also present a serialization layer to contend with
• A limited assembly load context model, explained in depthhere, that has a fixed set of assembly load contexts, each with their own behaviour:
  • The default load context, where assemblies are loaded by default
  • The load-from context, for loading assemblies manually at runtime
  • The reflection-only context, for safely loading assemblies to read their metadata without running them
  • The mysterious void that assemblies loaded with Assembly.LoadFile(string path) and Assembly.Load(byte[] asmBytes) live in

.NET Core (and .NET 5+) has eschewed this complexity for a simpler model:

• No Global Assembly Cache; applications bring all their own dependencies (PowerShell, as the plugin host, complicates this slightly for modules ??? its dependencies in $PSHOME are shared with all modules). This removes an exogenous factor for dependency resolution in applications, making dependency resolution more reproducible.
• Only one Application Domain, and no ability to create new ones. The Application Domain concept lives on purely to be the global state of the .NET process.
• A new, extensible Assembly Load Context model, where assembly resolution can be effectively namespaced by putting it in a new ALC. .NET Core processes begin with a single default ALC into which all assemblies are loaded (except for those loaded with Assembly.LoadFile(string) and Assembly.Load(byte[]) ), but are free to create and define their own custom ALCs with their own loading logic. When an assembly is loaded, the first ALC it is loaded into is responsible for resolving its dependencies, creating opportunities to create powerful .NET plugin loading mechanisms.

In both implementations, assemblies are loaded lazily when a method requiring their type is run for the first time.

For example here are two versions of the same code that load a dependency at different times.

The first will always load its dependency when Program.GetRange() is called, because the dependency reference is lexically present within the method:

using Dependency.Library;

public static class Program
{
    public static List<int> GetRange(int limit)
    {
        var list = new List<int>();
        for (int i = 0; i < limit; i++)
        {
            if (i >= 20)
            {
                // Dependency.Library will be loaded when GetNumbers is run
                // because the dependency call occurs directly within the method
                DependencyApi.Use();
            }

            list.Add(i);
        }
        return list;
    }
}

The second will load its dependency only if the limit parameter is 20 or more, because of the internal indirection through a method:

using Dependency.Library;

public static class Program
{
    public static List<int> GetNumbers(int limit)
    {
        var list = new List<int>();
        for (int i = 0; i < limit; i++)
        {
            if (i >= 20)
            {
                // Dependency.Library is only referenced within
                // the UseDependencyApi() method,
                // so will only be loaded when limit >= 20
                UseDependencyApi();
            }

            list.Add(i);
        }
        return list;
    }

    private static void UseDependencyApi()
    {
        // Once UseDependencyApi() is called, Dependency.Library is loaded
        DependencyApi.Use();
    }
}

This is a good thing for .NET to do, since it minimizes the memory and filesystem reads it needs to use, making it more resource efficient.

Unfortunately a side effect of this is that if an assembly load will fail, we won’t necessarily know when the program is first loaded, only when the code path that tries to load the assembly is run.

It can also set up timing conditions for assembly load conflicts; if two parts of the same program will try to load different versions of the same assembly, which one is loaded depends on which code path is run first.

For PowerShell this means that the following factors can affect an assembly load conflict:

• Which module was loaded first?
• Was the cmdlet/code path that uses the dependency library run?
• Does PowerShell load a conflicting dependency at startup or only under certain code paths?

Quick fixes and their limitations

In some cases it’s possible to make small adjustments to your module and fix things with a minimum of fuss. But these solutions tend to come with caveats, so that while they may apply to your module, they won’t work for every module.

Change your dependency version

The simplest way to avoid dependency conflicts is to agree on a dependency. This may be possible when:

• Your conflict is with a direct dependency of your module and you control the version
• Your conflict is with an indirect dependency, but you can configure your direct dependencies to use a workable indirect dependency version
• You know the conflicting version and can rely on it not changing

A good example of this last scenario is with the Newtonsoft.Json package. This is a dependency of PowerShell 6 and above, and isn’t used in Windows PowerShell, meaning a simple way to resolve versioning conflicts is to target the lowest version of Newtonsoft.Json across the PowerShell versions you wish to target.

For example, PowerShell 6.2.6 and PowerShell 7.0.2 both currently use Newtonsoft.Json version 12.0.3. So, to create a module targeting Windows PowerShell, PowerShell 6 and PowerShell 7, you would target Newtonsoft.Json 12.0.3 as a dependency and include it in your built module. When the module is loaded in PowerShell 6 or 7, PowerShell’s own Newtonsoft.Json assembly will already be loaded, but the version will be at least the required one for your module, meaning resolution will succeed. In Windows PowerShell, the assembly will not be already present in PowerShell, and so will be loaded from your module instead.

Generally, when targeting a concrete PowerShell package, like Microsoft.PowerShell.Sdk or System.Management.Automation, NuGet should be able to resolve the right dependency versions required. It’s only in the case of targeting both Windows PowerShell and PowerShell 6+ that dependency versioning becomes more difficult, either because of needing to target multiple frameworks, or due to targeting PowerShellStandard.Library.

Circumstances where pinning to a common dependency version won’t work include:

• The conflict is with an indirect dependency, and there’s no configuration of your dependencies that can use the common version
• The other dependency version is likely to change often, meaning settling on a common version will only be a short-term fix

Add an AssemblyResolve event handler to create a dynamic binding redirect

When changing your own dependency version isn’t possible, or is too hard, another way to make your module play nicely with other dependencies is to set up a dynamic binding redirect by registering an event handler for the AssemblyResolve event.

As with a static binding redirect, the point here to is force all consumers of a dependency to use the same actual assembly. This means we need to intercept calls to load the dependency and always redirect them to the version we want.

This looks something like this:

// Register the event handler as early as you can in your code.
// A good option is to use the IModuleAssemblyInitializer interface
// that PowerShell provides to run code early on when your module is loaded.

// This class will be instantiated on module import and the OnImport() method run.
// Make sure that:
//  - the class is public
//  - the class has a public, parameterless constructor
//  - the class implements IModuleAssemblyInitializer
public class MyModuleInitializer : IModuleAssemblyInitializer
{
    public void OnImport()
    {
        AppDomain.CurrentDomain.AssemblyResolve += DependencyResolution.ResolveNewtonsoftJson;
    }
}

// Clean up the event handler when the the module is removed
// to prevent memory leaks.
//
// Like IModuleAssemblyInitializer, IModuleAssemblyCleanup allows
// you to register code to run when a module is removed (with Remove-Module).
// Make sure it is also public with a public parameterless contructor
// and implements IModuleAssemblyCleanup.
public class MyModuleCleanup : IModuleAssemblyCleanup
{
    public void OnRemove()
    {
        AppDomain.CurrentDomain.AssemblyResolve -= DependencyResolution.ResolveNewtonsoftJson;
    }
}

internal static class DependencyResolution
{
    private static readonly string s_modulePath = Path.GetDirectoryName(
        Assembly.GetExecutingAssembly().Location);

    public static Assembly ResolveNewtonsoftJson(object sender, ResolveEventArgs args)
    {
        // Parse the assembly name
        var assemblyName = new AssemblyName(args.Name);

        // We only want to handle the dependency we care about.
        // In this example it's Newtonsoft.Json.
        if (!assemblyName.Name.Equals("Newtonsoft.Json"))
        {
            return null;
        }

        // Generally the version of the dependency you want to load is the higher one,
        // since it's the most likely to be compatible with all dependent assemblies.
        // The logic here assumes our module always has the version we want to load.
        // Also note the use of Assembly.LoadFrom() here rather than Assembly.LoadFile().
        return Assembly.LoadFrom(Path.Combine(s_modulePath, "Newtonsoft.Json.dll"));
    }
}

Note that you can technically register a scriptblock within PowerShell to handle an AssemblyResolve event, but:

AssemblyResolve

There are situations where redirecting to a common version will not work:

AssemblyResolve

Use the dependency out of process

This solution is more for module users than module authors, but is possible a solution to use when confronted with a module that won’t work due to an existing dependency conflict.

Dependency conflicts occur because two versions of the same assembly are loaded into the same .NET process . So a simple solution is to load them into different processes, as long as you can still use the functionality from both together.

In PowerShell, there are several ways to achieve this.

Invoke PowerShell as a subprocess

A simple way to run a PowerShell command out of the current process is to just start a new PowerShell process directly with the command call:

pwsh -c 'Invoke-ConflictingCommand'

The main limitation here is that restructuring the result can be trickier or more error prone than other options.

The PowerShell job system

The PowerShell job system also runs commands out of process, by sending commands to a new PowerShell process and returning the results:

$result = Start-Job { Invoke-ConflictingCommand } | Receive-Job -Wait

In this case, you just need to be sure that any variables and state are passed in correctly.

The job system can also be slightly cumbersome when running small commands.

PowerShell remoting

When it’s available, PowerShell remoting can be a very ergonomic way to run commands out of process. With remoting you can create a fresh PSSession in a new process, call its commands over PowerShell remoting and then use the results locally with, for example, the other module with the conflicting dependencies.

An example might look like this:

# Create a local PowerShell session
# where the module with conflicting assemblies will be loaded
$s = New-PSSession

# Import the module with the conflicting dependency via remoting,
# exposing the commands locally
Import-Module -PSSession $s -Name ConflictingModule

# Run a command from the module with the conflicting dependencies
Invoke-ConflictingCommand

Implicit remoting to Windows PowerShell

Another option in PowerShell 7 is to use the -UseWindowsPowerShell flag on Import-Module . This will import the module through a local remoting session into Windows PowerShell:

Import-Module -Name ConflictingModule -UseWindowsPowerShell

Be aware of course that modules may not work with or work differently with Windows PowerShell.

When not to use out-of-process invocation

As a module author, out-of-process command invocation is harder to bake into a module, and may have edge cases that cause issues. In particular, remoting and jobs may not be available in all environments where your module needs to work. However, the general principle of moving the implementation out of process and allowing the PowerShell module to be a thinner client, may still be applicable.

Of course, for module users too there will be cases where out-of-process invocation won’t work:

psobject

More robust solutions

The solutions above have all had the issue that there are scenarios and modules for which they won’t work. However, they also have the virtue of being relatively simple to implement correctly.

These next solutions we discuss are generally more robust, but also take somewhat more work to implement correctly and can introduce subtle bugs if not written carefully.

Loading through .NET Core Assembly Load Contexts

Assembly Load Contexts (ALCs) as an implementable API were introduced in .NET Core 1.0 to specifically address the need to load multiple versions of the same assembly into the same runtime.

Within .NET Core and .NET 5, they offer what is far and away the most robust solution to the problem of needing to load conflicting versions of an assembly. However, custom ALCs are not available in .NET Framework. This means that this solution will only work in PowerShell 6 and above.

Currently, the best example of using an ALC for dependency isolation in PowerShell is in PowerShell Editor Services (the language server for the PowerShell extension for Visual Studio Code), where an ALC is used to prevent PowerShell Editor Services’ own dependencies from clashing with those in PowerShell modules.

Implementing module dependency isolation with an ALC is conceptually difficult, but we will work through a minimal example here.

Imagine we have a simple module, only intended to work in PowerShell 7, whose source is laid out like this:

+ AlcModule.psd1
+ src/
    + TestAlcModuleCommand.cs
    + AlcModule.csproj

The cmdlet implementation looks like this:

using Shared.Dependency;

namespace AlcModule
{
    [Cmdlet(VerbsDiagnostic.Test, "AlcModule")]
    public class TestAlcModuleCommand : Cmdlet
    {
        protected override void EndProcessing()
        {
            // Here's where our dependency gets used
            Dependency.Use();
            // Something trivial to make our cmdlet do *something*
            WriteObject("done!");
        }
    }
}

The (heavily simplified) manifest, looks like this:

@{
    Author = 'Me'
    ModuleVersion = '0.0.1'
    RootModule = 'AlcModule.dll'
    CmdletsToExport = @('Test-AlcModule')
    PowerShellVersion = '7.0'
}

And the csproj looks like this:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netcoreapp3.1</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Shared.Dependency" Version="1.0.0" />
    <PackageReference Include="Microsoft.PowerShell.Sdk" Version="7.0.1" PrivateAssets="all" />
  </ItemGroup>
</Project>

When we build this module, the generated output has the following layout:

AlcModule/
  + AlcModule.psd1
  + AlcModule.dll
  + Shared.Dependency.dll

In this example, our problem lies in the Shared.Dependency.dll assembly, which is our imaginary conflicting dependency. This is the dependency that we need to put behind an ALC so that we can use the module specific one.

We seek to re-engineer the module so that:

• Module dependencies are only ever loaded into our custom ALC, and not into PowerShell’s ALC, so there can be no conflict. Moreover, as we add more dependencies to our project, we don’t want to continuously add more code to keep loading working; instead we want reusable, generic dependency resolution logic.
• Loading the module will still work as normal in PowerShell, meaning cmdlets and other types the PowerShell module system needs to see will be defined within PowerShell’s own ALC.

To mediate these two requirements, we must break our module up into two assemblies:

• A cmdlets assembly, AlcModule.Cmdlets.dll, which will contain definitions of all the types that PowerShell’s module system needs to load the module correctly. Namely any implementations of the Cmdlet base class and our IModuleAssemblyInitializer -implementing class, which will set up the event handler for AssemblyLoadContext.Default.Resolving to properly load AlcModule.Engine.dll through our custom ALC. Any types that are meant to be publicly exposed to PowerShell must also be defined here, since PowerShell 7 deliberately hides types defined in assemblies loaded in other ALCs. Finally, this assembly will also need to be where our custom ALC definition lives. Beyond that, as little code should live in this as possible.
• An engine assembly, AlcModule.Engine.dll, which handles all the actual implementation of the module. Types from this will still be available in the PowerShell ALC, but it will initially be loaded through our custom ALC, and its dependencies will only ever be loaded into the custom ALC. Effectively, this becomes a “bridge” between the two ALCs.

Using this bridge concept, our new assembly situation effectively will look like this:

To make sure the default ALC’s dependency probing logic will not resolve the dependencies to be loaded into the custom ALC, we will need to separate these two parts of the module in different directories. So our new module layout will look like this:

AlcModule/
  AlcModule.Cmdlets.dll
  AlcModule.psd1
  Dependencies/
  | + AlcModule.Engine.dll
  | + Shared.Dependency.dll

To see how our implementation now changes, we’ll start with the implementation of AlcModule.Engine.dll:

using Shared.Dependency;

namespace AlcModule.Engine
{
    public class AlcEngine
    {
        public static void Use()
        {
            Dependency.Use();
        }
    }
}

This is a fairly straightforward container for the dependency, Shared.Dependency.dll, but you should think of it as the .NET API for your functionality, which the cmdlets living in the other assembly will wrap for PowerShell.

The cmdlet in AlcModule.Cmdlets.dll now looks like this:

// Reference our module's Engine implementation here
using AlcModule.Engine;

namespace AlcModule.Cmdlets
{
    [Cmdlet(VerbsDiagnostic.Test, "AlcModule")]
    public class TestAlcModuleCommand : Cmdlet
    {
        protected override void EndProcessing()
        {
            AlcEngine.Use();
            WriteObject("done!");
        }
    }
}

At this point, if we were to load AlcModule and run Test-AlcModule, we would hit a FileNotFoundException when the default ALC tries to load Alc.Engine.dll to run EndProcessing() . This is good, since it means the default ALC can’t find the dependencies we want to hide.

So now we need to add the magic to AlcModule.Cmdlets.dll that helps it know how to resolve AlcModule.Engine.dll. First we must define our custom ALC that will resolve assemblies from our module’s Dependencies directory:

namespace AlcModule.Cmdlets
{
    internal class AlcModuleAssemblyLoadContext : AssemblyLoadContext
    {
        private readonly string _dependencyDirPath;

        public AlcModuleAssemblyLoadContext(string dependencyDirPath)
        {
            _depdendencyDirPath = dependencyDirPath;
        }

        protected override Assembly Load(AssemblyName assemblyName)
        {
            // We do the simple logic here of
            // looking for an assembly of the given name
            // in the configured dependency directory
            string assemblyPath = Path.Combine(
                s_dependencyDirPath,
                $"{assemblyName.Name}.dll");

            // The ALC must use inherited methods to load assemblies
            // Assembly.Load*() won't work here
            return LoadFromAssemblyPath(assemblyPath);
        }
    }
}

Then, we need to hook our custom ALC up to the default ALC’s Resolving event (which is the ALC version of the AssemblyResolve event on Application Domains) that will be fired when EndProcessing() is called to try and find AlcModule.Engine.dll:

namespace AlcModule.Cmdlets
{
    public class AlcModuleResolveEventHandler : IModuleAssemblyInitializer, IModuleAssemblyCleanup
    {
        // Get the path of the dependency directory.
        // In this case we find it relative to the AlcModule.Cmdlets.dll location
        private static readonly string s_dependencyDirPath = Path.GetFullPath(
            Path.Combine(
                Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location),
                "Dependencies"));

        private static readonly AlcModuleAssemblyLoadContext s_dependencyAlc = new AlcModuleAssemblyLoadContext(s_dependencyDirPath);

        public void OnImport()
        {
            // Add the Resolving event handler here
            AssemblyLoadContext.Default.Resolving += ResolveAlcEngine;
        }

        public void OnRemove()
        {
          // Remove the Resolving event handler here
          AssemblyLoadContext.Default.Resolving -= ResolveAlcEngine;
        }

        private static Assembly ResolveAlcEngine(
            AssemblyLoadContext defaultAlc,
            AssemblyName assemblyToResolve)
        {
            // We only want to resolve the Alc.Engine.dll assembly here.
            // Because this will be loaded into the custom ALC,
            // all of *its* dependencies will be resolved
            // by the logic we defined for that ALC's implementation.
            //
            // Note that we are safe in our assumption that the name is enough
            // to distinguish our assembly here,
            // since it's unique to our module.
            // There should be no other AlcModule.Engine.dll on the system.
            if (!assemblyToResolve.Name.Equals("AlcModule.Engine"))
            {
                return null;
            }

            // Allow our ALC to handle the directory discovery concept
            //
            // This is where Alc.Engine.dll is loaded into our custom ALC
            // and then passed through into PowerShell's ALC,
            // becoming the bridge between both
            return s_dependencyAlc.LoadFromAssemblyName(assemblyToResolve);
        }
    }
}

With the new implementation laid out, we can now take a look at the sequence of calls that occurs when the module is loaded and Test-AlcModule is run:

Some points of interest are:

• The IModuleAssemblyInitializer is run first when the module loads and sets the Resolving event
• We don’t even load the dependencies until Test-AlcModule is run and its EndProcessing() method is called
• When EndProcessing() is called, the default ALC does not find AlcModule.Engine.dll and fires the Resolving event
• Our event handler here is what does the magic of hooking up the custom ALC to the default ALC, and only loads AlcModule.Engine.dll
• Then, within AlcModule.Engine.dll, when AlcEngine.Use() is called, the custom ALC again kicks in to resolve Shared.Dependency.dll. Specifically it always loads our Shared.Dependency.dll, since it never conflicts with anything in the default ALC and only looks in our Dependencies directory.

Assembling the implementation, our new source code layout looks like this:

+ AlcModule.psd1
+ src/
  + AlcModule.Cmdlets/
  | + AlcModule.Cmdlets.csproj
  | + TestAlcModuleCommand.cs
  | + AlcModuleAssemblyLoadContext.cs
  | + AlcModuleInitializer.cs
  |
  + AlcModule.Engine/
  | + AlcModule.Engine.csproj
  | + AlcEngine.cs

AlcModule.Cmdlets.csproj looks like:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netcoreapp3.1</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <ProjectReference Include="..\AlcModule.Engine\AlcModule.Engine.csproj" />
    <PackageReference Include="Microsoft.PowerShell.Sdk" Version="7.0.1" PrivateAssets="all" />
  </ItemGroup>
</Project>

AlcModule.Engine.csproj looks like this:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netcoreapp3.1</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Shared.Dependency" Version="1.0.0" />
  </ItemGroup>
</Project>

And so when we build the module, our strategy is:

• Build AlcModule.Engine
• Build AlcModule.Cmdlets
• Copy everything from AlcModule.Engine into the Dependencies dir, and remember what we copied
• Copy everything from AlcModule.Cmdlets that wasn’t in AlcModule.Engine into the base module dir

Since the module layout here is so crucial to dependency separation, here’s a build script to use from the source root:

param(
    # The .NET build configuration
    [ValidateSet('Debug', 'Release')]
    [string]
    $Configuration = 'Debug'
)

# Convenient reusable constants
$mod = "AlcModule"
$netcore = "netcoreapp3.1"
$copyExtensions = @('.dll', '.pdb')

# Source code locations
$src = "$PSScriptRoot/src"
$engineSrc = "$src/$mod.Engine"
$cmdletsSrc = "$src/$mod.Cmdlets"

# Generated output locations
$outDir = "$PSScriptRoot/out/$mod"
$outDeps = "$outDir/Dependencies"

# Build AlcModule.Engine
Push-Location $engineSrc
dotnet publish -c $Configuration
Pop-Location

# Build AlcModule.Cmdlets
Push-Location $cmdletsSrc
dotnet publish -c $Configuration
Pop-Location

# Ensure out directory exists and is clean
Remove-Item -Path $outDir -Recurse -ErrorAction Ignore
New-Item -Path $outDir -ItemType Directory
New-Item -Path $outDeps -ItemType Directory

# Copy manifest
Copy-Item -Path "$PSScriptRoot/$mod.psd1"

# Copy each Engine asset and remember it
$deps = [System.Collections.Generic.Hashtable[string]]::new()
Get-ChildItem -Path "$engineSrc/bin/$Configuration/$netcore/publish/" |
    Where-Object { $_.Extension -in $copyExtensions } |
    ForEach-Object { [void]$deps.Add($_.Name); Copy-Item -Path $_.FullName -Destination $outDeps }

# Now copy each Cmdlets asset, not taking any found in Engine
Get-ChildItem -Path "$cmdletsSrc/bin/$Configuration/$netcore/publish/" |
    Where-Object { -not $deps.Contains($_.Name) -and $_.Extension -in $copyExtensions } |
    ForEach-Object { Copy-Item -Path $_.FullName -Destination $outDir }

So finally, we have a general way to use an Assembly Load Context to isolate our module’s dependencies that remains robust over time and as more dependencies are added.

Hopefully this example is informative, but naturally a fuller example would be more helpful. For that, go to this GitHub repository I created to give a full demonstration of how to migrate a module to use an ALC, while keeping that module working in .NET Framework and also using .NET Standard and PowerShell Standard to simply the core implementation.

Assembly resolve handler for side-by-side loading with Assembly.LoadFile()

Another, possibly simpler, way to achieve side-by-side assembly loading is to hook up an AssemblyResolve event to Assembly.LoadFile() . Using Assembly.LoadFile() has the advantage of being the simplest solution to implement and working with both .NET Core and .NET Framework.

To show this, let’s take a look at a quick example of an implementation that combines ideas from our dynamic binding redirect example, and from the Assembly Load Context example above.

We’ll call this module LoadFileModule , which has a trival command Test-LoadFile [-Path] <path> . This module takes a dependency on the CsvHelper assembly (version 15.0.5).

As with the ALC module, we must first split up the module into two pieces. First the part that does the actual implementation, LoadFileModule.Engine :

using CsvHelper;

namespace LoadFileModule.Engine
{
    public class Runner
    {
        public void Use(string path)
        {
            // Here's where we use the CsvHelper dependency
            using (var reader = new StreamReader(path))
            using (var csvReader = new CsvReader(reader, CultureInfo.InvariantCulture))
            {
                // Imagine we do something useful here...
            }
        }
    }
}

And then the part that PowerShell will load directly, LoadFileModule.Cmdlets . We use a very similar strategy as with the ALC module, where we isolate dependencies in a separate directory. But this time we must load all assemblies in with a resolve event, rather than just LoadFileModule.Engine.dll.

using LoadFileModule.Engine;

namespace LoadFileModule.Cmdlets
{
    // Actual cmdlet definition
    [Cmdlet(VerbsDiagnostic.Test, "LoadFile")]
    public class TestLoadFileCommand : Cmdlet
    {
        [Parameter(Mandatory = true)]
        public string Path { get; set; }

        protected override void EndProcessing()
        {
            // Here's where we use LoadFileModule.Engine
            new Runner().Use(Path);
        }
    }

    // This class controls our dependency resolution
    public class ModuleContextHandler : IModuleAssemblyInitializer, IModuleAssemblyCleanup
    {
        // We catalog our dependencies here to ensure we don't load anything else
        private static IReadOnlyDictionary<string, int> s_dependencies = new Dictionary<string, int>
        {
            { "CsvHelper", 15 },
        };

        // Set up the path to our dependency directory within the module
        private static string s_dependenciesDirPath = Path.GetFullPath(
            Path.Combine(
                Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location),
                "Dependencies"));

        // This makes sure we only try to resolve dependencies when the module is loaded
        private static bool s_engineLoaded = false;

        public void OnImport()
          {
            // Set up our event when the module is loaded
            AppDomain.CurrentDomain.AssemblyResolve += HandleResolveEvent;
        }

        public void OnRemove(PSModuleInfo psModuleInfo)
        {
            // Unset the event when the module is unloaded
            AppDomain.CurrentDomain.AssemblyResolve -= HandleResolveEvent;
        }

        private static Assembly HandleResolveEvent(object sender, ResolveEventArgs args)
        {
            var asmName = new AssemblyName(args.Name);

            // First we want to know if this is the top-level assembly
            if (asmName.Name.Equals("LoadFileModule.Engine"))
            {
                s_engineLoaded = true;
                return Assembly.LoadFile(Path.Combine(s_dependenciesDirPath, "Unrelated.Engine.dll"));
            }

            // Otherwise, if that assembly has been loaded, we must try to resolve its dependencies too
            if (s_engineLoaded
                && s_dependencies.TryGetValue(asmName.Name, out int requiredMajorVersion)
                && asmName.Version.Major == requiredMajorVersion)
            {
                string asmPath = Path.Combine(s_dependenciesDirPath, $"{asmName.Name}.dll");
                return Assembly.LoadFile(asmPath);
            }

            return null;
        }
    }
}

Finally, the layout of the module is also similar to the ALC module:

LoadFileModule/
  + LoadFileModule.Cmdlets.dll
  + LoadFileModule.psd1
  + Dependencies/
  |  + LoadFileModule.Engine.dll
  |  + CsvHelper.dll

With this structure in place, LoadFileModule now supports being loaded alongside other modules with a dependency on CsvHelper.

Because our handler will apply to all AssemblyResolve events across the Application Domain, we’ve needed to make some specific design choices here:

• We only start handling general dependency loading after LoadFileModule.Engine.dll has been loaded, to narrow the window in which our event may interfere with other loading.
• We push LoadFileModule.Engine.dll out into the Dependencies directory, so that it’s loaded by a LoadFile() call rather than by PowerShell. This means its own dependency loads will always raise an AssemblyResolve event, even if another CsvHelper.dll (for example) is loaded in PowerShell, meaning we have an opportunity to find the correct dependency.
• We are forced to code a dictionary of dependency names and versions into our handler, since we must try only to resolve our specific dependencies for our module. In our particular case, it would be possible to use the ResolveEventArgs.RequestingAssembly property to work out whether CsvHelper is being requested by our module, but this wouldn’t work for dependencies of dependencies (for example if CsvHelper itself raised an AssemblyResolve event). There are other, harder ways to do this, such as comparing requested assemblies to the metadata of assemblies in the Dependencies directory, but here we’ve made this checking more explicit both to highlight the issue and to simplify the example.

Essentially this is the key difference between the LoadFile() strategy and the ALC strategy: the AssemblyResolve event must service all loads in the Application Domain, making it much harder to keep our dependency resolution from being tangled with other modules. However, the lack of ALCs in .NET Framework makes this option one worth understanding (even just for .NET Framework, while using an ALC in .NET Core).

Custom Application Domains

A final (and by some measures, extreme) option for assembly isolation is custom Application Domain use. Application Domains (used in the plural), are only available in .NET Framework, and are used to provide in-process isolation between parts of a .NET application. One of the uses of this is to isolate assembly loads from each other within the same process.

However, Application Domains are serialization boundaries, meaning that objects in one Application Domain cannot be referenced and used directly by objects in another Application Domain. This can be worked around by implementing MarshalByRefObject , but when you don’t control the types (as is often the case with dependencies), it’s not possible to force an implementation here, meaning that the only solution is to make large architectural changes. The serialization boundary also has serious performance implications.

Because Application Domains have this serious limitation, are complicated to implement, and only work in .NET Framework, I won’t give an example of how you might use them here. While they’re worth mentioning as a possibility, they aren’t an investment I would recommend.

If you’re interested in trying to use a custom Application Domain, the following links might help:

• Conceptual documentation on Appliation Domains
• Examples for using Application Domains

Solutions for dependency conflicts that don’t work for PowerShell

Finally, I should address some possibilities that come up when researching .NET dependency conflicts in .NET that can look promising but generally won’t work for PowerShell.

These solutions have the common theme that they are changes in deployment configurations in an environment where you control the application and possibly the entire machine. This is because they are oriented toward scenarios like web servers and other applications deployed to server environments, where the environment is intended to host the application and is free to be configured by the deploying user.

They also tend to be very much .NET Framework focused, meaning they will not work with PowerShell 6 or above.

Note that if you know that your module will only be used in environments you have total control over, and only with Windows PowerShell 5.1 and below, some of these may be options.

In general however, modules should not modify global machine state like this , as it can break configurations, causing powershell.exe, other modules or other dependent applications not to work, or simply fail and cause your module to fail in unexpected ways.

Static binding redirect with app.config to force using the same dependency version

.NET Framework applications can take advantage of an app.config file to configure some of the application’s behaviours declaratively. It’s possible to write an app.config entry that configures assembly binding to redirect assembly loading to a particular version.

Two issues with this for PowerShell are:

• .NET Core does not support app.config , so this is a powershell.exe -only solution.
• powershell.exe is a shared application that lives under the System32 directory. This means its likely your module won’t be able to modify its contents on many systems, and even if it can, modifying the app.config could break an existing configuration or affect the loading of other modules.

Setting codebase with app.config

For the same reasons as with setting binding redirects in app.config , trying to configure the app.config codebase setting is generally not going to work in PowerShell modules.

Installing your dependencies to the Global Assembly Cache (GAC)

Another way to resolve dependency version conflicts in .NET Framework is to install dependencies to the GAC, so that different versions can be loaded side-by-side from the GAC.

Again for PowerShell modules, the chief issues here are:

• The GAC only applies to .NET Framework, so this will not help in PowerShell 6 and above.
• Installing assemblies to the GAC is a modification of global machine state and may cause side-effects in other applications or to other modules. It may also be difficult to do correctly, even when your module has the required access privileges (and getting it wrong could cause serious issues in other .NET applications machine-wide).

Solving the issue in PowerShell itself…

After reading through all of this and seeing the complexity not just of implementing an isolation solution, but making it work with the PowerShell module system, you may wonder why PowerShell hasn’t put a solution to this problem into the module system yet.

While it’s not something we’re planning to implement imminently, the facility of Assembly Load Contexts available in .NET 5 makes this something worth considering long term.

However, this would represent a large change in the module system, which is a very critical (and already complex) component of PowerShell. In addition, the diversity of PowerShell modules and module scenarios presents a serious challenge in terms of PowerShell correctly isolating modules consistently and without creating edge cases.

In particular, dependencies will be exposed if they are part of a module’s API. For example, if a PowerShell module converts YAML strings to objects and uses YamlDotNet to return objects of type YamlDotNet.YamlDocument , then its dependencies are exposed to the global context.

This can lead to type identity issues when instances of the exposed type are used with APIs expecting the same type (but from a different assembly); specifically you may see confusing messages like “Type YamlDotNet.YamlDocument cannot be cast to type YamlDotNet.YamlDocument “, because even though the two types have the same name, .NET regards them as coming from different assemblies and therefore being different.

In order to safely isolate a dependency used in the module API, API types need to either be a type already found in PowerShell and its public dependencies, like PSObject or System.Xml.XmlDocument , or a new type defined by the module to be an intermediary between the PowerShell context and the dependency context.

It’s likely that any module isolation strategy could break some module, and that module authors will need, to some extent, to understand the implications of dependency isolation and be responsible for deciding whether their module can have its dependencies isolated.

So with that in mind, I encourage you to contribute to the discussion on GitHub . Also take a look at the RFC proposed to address the issue.

Further reading

There’s plenty more to read on the topic of .NET assembly version dependency conflicts. Here are some nice jumping off points:

• .NET: Assemblies in .NET
• .NET Core: The managed assembly loading algorithm
• .NET Core: Understanding System.Runtime.Loader.AssemblyLoadContext
• .NET Core: Discussion about side-by-side assembly loading solutions
• .NET Framework: Redirecting assembly versions
• .NET Framework: Best practices for assembly loading
• .NET Framework: How the runtime locates assemblies
• .NET Framework: Resolve assembly loads
• StackOverflow: Assembly binding redirect, how and why?
• PowerShell: Discussion about implementing AssemblyLoadContexts
• PowerShell: Assembly.LoadFile() doesn’t load into default AssemblyLoadContext
• Rick Strahl: When does a .NET assembly dependency get loaded?
• Jon Skeet: Summary of versioning in .NET
• Nate McMaster: Deep dive into .NET Core primitives

Final notes

In this blog post, we looked over various ways to solve the issue of having module dependency conflicts in PowerShell, identifying strategies that won’t work, simple strategies that sometimes work, and more complex strategies that are more robust.

In particular we looked at how to implement an Assembly Load Context in .NET Core to make module dependency isolation much easier in PowerShell 6 and up.

This is fairly complicated subject matter, so don’t worry if it doesn’t click immediately. Feel free to read the material here and linked, experiment with implementations (try stepping through it in a debugger), and also get in touch with us on GitHub or Twitter.

Cheers!

Rob Holt

Software Engineer

PowerShell Team