DbContext recreate with cache clear
-
I'm trying to get my web application to reload a DbContext cache after initialization to introduce new entities (if available, as introduced via MEF) into the context model. Currently the context populates via an OnModelCreating overload.
public class MyContext : DbContext
{
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
var models = DependencyResolver.Current.GetServices(typeof(IModel));
foreach (var model in models)
{
var entityMethod = typeof(DbModelBuilder).GetMethod("Entity");
entityMethod.MakeGenericMethod(model.GetType())
.Invoke(modelBuilder, new object[] { });
}
}I use a custom initializer that will add entities as appropriate on initilization ( I'm working on an archive for entity removal, but that's another topic). My issue is this: I would like to dynamically load modules, have the context pick-up the IModel objects immediately and make them available for use. Does anyone know of a hook that will allow me to re-initialize the context and clear the cache without a complete application restart? Note: I don't want to disable the cache completely, as this will not be an overly common task and it does offer more benefit than this functionality is worth.
-
I'm trying to get my web application to reload a DbContext cache after initialization to introduce new entities (if available, as introduced via MEF) into the context model. Currently the context populates via an OnModelCreating overload.
public class MyContext : DbContext
{
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
var models = DependencyResolver.Current.GetServices(typeof(IModel));
foreach (var model in models)
{
var entityMethod = typeof(DbModelBuilder).GetMethod("Entity");
entityMethod.MakeGenericMethod(model.GetType())
.Invoke(modelBuilder, new object[] { });
}
}I use a custom initializer that will add entities as appropriate on initilization ( I'm working on an archive for entity removal, but that's another topic). My issue is this: I would like to dynamically load modules, have the context pick-up the IModel objects immediately and make them available for use. Does anyone know of a hook that will allow me to re-initialize the context and clear the cache without a complete application restart? Note: I don't want to disable the cache completely, as this will not be an overly common task and it does offer more benefit than this functionality is worth.
You picked a very odd place to put the cache initialization code. You also don't mention what kind of application this is, Windows Forms, WPF, or ASP.NET. It would be better if you implemented your own caching scheme so you have more flexibility on when it initializes, what it loads and how and what you clear. Your code implies that you're relying on a context that is created once during application startup, is never disposed and is basically pre-loading the entire database into memory. That really doesn't scale well! I did something similar in a web app I'm working on now. I created my own custom caching scheme. It's just a single interface and a couple of classes really, and based all of my controllers and service layers on base implementations of it. Data items that are used frequently, but rarely ever change are cached only when they are initially requested and dumped and reloaded at various points in the code when required, depending on what edit operations are taking place. I can also either dump an entire table of cached data or just a single item and replace it.
A guide to posting questions on CodeProject
How to debug small programs
Dave Kreskowiak -
You picked a very odd place to put the cache initialization code. You also don't mention what kind of application this is, Windows Forms, WPF, or ASP.NET. It would be better if you implemented your own caching scheme so you have more flexibility on when it initializes, what it loads and how and what you clear. Your code implies that you're relying on a context that is created once during application startup, is never disposed and is basically pre-loading the entire database into memory. That really doesn't scale well! I did something similar in a web app I'm working on now. I created my own custom caching scheme. It's just a single interface and a couple of classes really, and based all of my controllers and service layers on base implementations of it. Data items that are used frequently, but rarely ever change are cached only when they are initially requested and dumped and reloaded at various points in the code when required, depending on what edit operations are taking place. I can also either dump an entire table of cached data or just a single item and replace it.
A guide to posting questions on CodeProject
How to debug small programs
Dave KreskowiakThe DbContext lives in an MVC application, but from my understanding that should be immaterial as it's specifically an EF implementation and not an MVC one. The context itself is wrapped by a separate UnitOfWork object that is injected into my controllers, and it explicitly disposes each context instance (per request). The UoW abstracts the data source so that I can use the same controller-base with multiple sorts of data sources (XML, Web services, LDAP, etc). I like your thought of a custom cache control; I've been so wrapped up in the details of the Entity Framework that such a clear solution hadn't crossed my mind, sadly enough. Forests and trees. I use that as a fallback if I cannot determine another implementation using the framework itself. Thank you for the perspective!
-
The DbContext lives in an MVC application, but from my understanding that should be immaterial as it's specifically an EF implementation and not an MVC one. The context itself is wrapped by a separate UnitOfWork object that is injected into my controllers, and it explicitly disposes each context instance (per request). The UoW abstracts the data source so that I can use the same controller-base with multiple sorts of data sources (XML, Web services, LDAP, etc). I like your thought of a custom cache control; I've been so wrapped up in the details of the Entity Framework that such a clear solution hadn't crossed my mind, sadly enough. Forests and trees. I use that as a fallback if I cannot determine another implementation using the framework itself. Thank you for the perspective!
MVC... OK. The problem with what you're doing is that all the EF data that you're pre-loading goes away on every request! Upon a request, your Controller gets created, which is creating a UnitOfWork, which is creating a context instance, which is loading all that data. When the request is done, the Controller Disposes, which also Disposes the UnitOfWork, which is also Disposing the DbContext. All that data goes with it. It's not retained in memory. You're not really caching anything at all, and , frankly, you're probably using up more time loading all the tables than if you just loaded the data the controller needs. EDIT: I may be misunderstanding what you're trying to do. Are you trying to modify the model on-the-fly and having EF rebuild itself based on the new model?
A guide to posting questions on CodeProject
How to debug small programs
Dave Kreskowiak -
MVC... OK. The problem with what you're doing is that all the EF data that you're pre-loading goes away on every request! Upon a request, your Controller gets created, which is creating a UnitOfWork, which is creating a context instance, which is loading all that data. When the request is done, the Controller Disposes, which also Disposes the UnitOfWork, which is also Disposing the DbContext. All that data goes with it. It's not retained in memory. You're not really caching anything at all, and , frankly, you're probably using up more time loading all the tables than if you just loaded the data the controller needs. EDIT: I may be misunderstanding what you're trying to do. Are you trying to modify the model on-the-fly and having EF rebuild itself based on the new model?
A guide to posting questions on CodeProject
How to debug small programs
Dave KreskowiakAccording to the MSDN[^] documentation the cache is created on first instantiation and tied to the AppDomain. To an extent that is my issue, because I cannot reset that cache. I can inject new IModels in at runtime and they are not included in new instances of the DbContext until the application is restarted, at which point OnModelCreating is called again. This implies to me that the cache is functioning as intended. The population of the context model is happening just once, as is efficient, but I would like to bump it when an extension is added to re-build the context model again. Fortunately the DbContext doesn't actually execute any SQL until an IQuryable has a terminal method run on it, such as .Single, .First, or .toList; I agree that it would be mind-numbingly inefficient if it did. Instead, after the initial context creation it automatically caches the specific implementation of DbContext so that it holds the ORM but not the table data. Future instances come from this cache instead of being generated at runtime. It's pretty slick, honestly, and does perform nicely. My only complaint is the completely crazy statements it generates for Table-Per-Type implementations (which model polymorphism much better than Table-Per-Hierarchy or Table-Per-Class) and, of course, this cache refresh obnoxiousness. I think I understand the communication breakdown here. I'm not interested in the data, I want to call that per request in accordance with the UnitOfWork pattern. I want to find that DbContext model cache and refresh it when a new extension is loaded.
-
According to the MSDN[^] documentation the cache is created on first instantiation and tied to the AppDomain. To an extent that is my issue, because I cannot reset that cache. I can inject new IModels in at runtime and they are not included in new instances of the DbContext until the application is restarted, at which point OnModelCreating is called again. This implies to me that the cache is functioning as intended. The population of the context model is happening just once, as is efficient, but I would like to bump it when an extension is added to re-build the context model again. Fortunately the DbContext doesn't actually execute any SQL until an IQuryable has a terminal method run on it, such as .Single, .First, or .toList; I agree that it would be mind-numbingly inefficient if it did. Instead, after the initial context creation it automatically caches the specific implementation of DbContext so that it holds the ORM but not the table data. Future instances come from this cache instead of being generated at runtime. It's pretty slick, honestly, and does perform nicely. My only complaint is the completely crazy statements it generates for Table-Per-Type implementations (which model polymorphism much better than Table-Per-Hierarchy or Table-Per-Class) and, of course, this cache refresh obnoxiousness. I think I understand the communication breakdown here. I'm not interested in the data, I want to call that per request in accordance with the UnitOfWork pattern. I want to find that DbContext model cache and refresh it when a new extension is loaded.
That makes things a bit clearer! OK, what you're refering to as a "cache" isn't. EF has to generate new models and mapping views of the model (query trees!) for query and updating as well as validate those views to make sure they make sense. This is what you're trying to get EF to regenerate. It's not really a cache but is the data EF uses to maintain the model and database mappings. EF does maintain caches, such as the first-level object cache, the query plan cache and metadata cache. If you change the model, these caches must also be dumped and rebuilt. And then there's are second-level caching that you provide. The problem with this is that EF maps strong types to the database using the Conceptual model. Without recompiling the code, you'd have to create/modify new entity classes on-the-fly in .NET to map them to the model. But, even then, once the changes are made, how are you going to maintain those changes if the app restarts?? I think I know what you're trying to do, but I don't know of any way to do it, let alone support it in a production environment. You're trying to out-smart Entity Framework, and frankly, the people that wrote it. I found a post 4 years ago by Julie Lerman that says she doesn't know of a way to do it and she's a developer on the EF code! But, there's also this blog post[^] that describes what might be a possible solution. The only thing I've found that might do what you want is an old project on CodePlex that hasn't been updated since 2009, EFModelAdapter[^] I have no idea if it works with anything other than EF4. Somehow, I doubt it will.
-
That makes things a bit clearer! OK, what you're refering to as a "cache" isn't. EF has to generate new models and mapping views of the model (query trees!) for query and updating as well as validate those views to make sure they make sense. This is what you're trying to get EF to regenerate. It's not really a cache but is the data EF uses to maintain the model and database mappings. EF does maintain caches, such as the first-level object cache, the query plan cache and metadata cache. If you change the model, these caches must also be dumped and rebuilt. And then there's are second-level caching that you provide. The problem with this is that EF maps strong types to the database using the Conceptual model. Without recompiling the code, you'd have to create/modify new entity classes on-the-fly in .NET to map them to the model. But, even then, once the changes are made, how are you going to maintain those changes if the app restarts?? I think I know what you're trying to do, but I don't know of any way to do it, let alone support it in a production environment. You're trying to out-smart Entity Framework, and frankly, the people that wrote it. I found a post 4 years ago by Julie Lerman that says she doesn't know of a way to do it and she's a developer on the EF code! But, there's also this blog post[^] that describes what might be a possible solution. The only thing I've found that might do what you want is an old project on CodePlex that hasn't been updated since 2009, EFModelAdapter[^] I have no idea if it works with anything other than EF4. Somehow, I doubt it will.
I've spent some time poking around the EF 6 code and it looks like all access to the Tuple where the DbCompiledModel is stored are internal. I could implement a cache for the DbCompiledModel, but the amount of work to get it all wired in for use seems a little oppressive. I finally settled on a basic HttpRuntime.UnloadAppDomain in the module-loading function to force a Context (and site) rebuild when new extensions are added. The user-level experience should be effectively the same, only I will know that it's not as snazzy as originally intended. I think that this might be a good off-time project, at least if I plan to make deadlines! By the way, the the Conceptual model is populated via the code snippet in the original question, which uses a DI provider (in this case a custom MEF IDependencyResolver) to determine the pool of entities required for tracking. The intent was that a DirectoryCatalog.Refresh() would automatically provide new entities for use without an application restart. It works fine for Controllers, ViewModels, and, of course, the Model classes themselves. Adding the Models to the context has been the only sticking point. Thanks again for the help, Mr. Kreskowiak.