I thought .NET was supposed to make things easier, if anything, than unmanaged code.
-
You do know about the
System.IO.MemoryMappedFilesnamespace, right? MemoryMappedFile Class (System.IO.MemoryMappedFiles) | Microsoft Docs[^] If you have a C++ version of your app, you could run them side-by-side and compare performance.".45 ACP - because shooting twice is just silly" - JSOP, 2010
-----
You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010
-----
When you pry the gun from my cold dead hands, be careful - the barrel will be very hot. - JSOP, 2013Yes, though it was added to .NET after my initial attempt at using mem mapped files from C#. Besides all that is is a wrapper like the one i had written years ago. It doesn't change the basic problem which is: var foo = new int[1000000]; //backed by disk, paged automatically, in C/C++ it's mainly because you can't use pointers in C#, and even if you use unsafe, you cannot pin objects to specific addresses in memory
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
-
What's the problem? Just do exactly that:
int[] foo = new int[1000000];
And it'll be allocated for you from the LOH. Heck, you can do this if you want to:
int[] foo = new int[500000000];
Provided the max index fits in 31 bits (and the whole item is less than 2GB) .NET will let you have it if it can.
Sent from my Amstrad PC 1640 Never throw anything away, Griff Bad command or file name. Bad, bad command! Sit! Stay! Staaaay... AntiTwitter: @DalekDave is now a follower!
i know that, but it's not efficient and doesn't solve the basic issue, which is persistent, rapidly searchable storage.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
-
I know this is a very special case but still i ran headlong into it. The easiest way to implement a B+ tree on disk is using a memory mapped file. I think this is what SQL Server does, but don't quote me. However, the only way you can access memory mapped files in C# is through .NET interop which makes it useless. Because one of the points of a memory mapped file is that you can do memory allocations that are backed by disk. There's no way in hell .NET can give you that in its current incarnation, even if one were to write a custom host, because of the way a GC system works. What I'd like var foo = new int[1000000]; // backed by disk, paged automatically What I'd have to do. somepointer = VirtualAlloc(...) Write(somepointer, data) etc etc basically it works like file i/o which defeats essentially the whole purpose. =(
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
-
Quote:
because of the way a GC system works
:-D :-D :-D[^].
Oh boy, you've been paying attention GC is great for string management compared to a traditional heap. for mapping a b-tree or b+tree to disk using memory mapped files not so much. It would be cool if .NET had a mechanism whereby you could create uncollected heaps that you manually destroy, and could allocate objects to them somehow. Maybe by making an appdomain with an UN GC'd heap in it or something. I know you can suspend garbage collection but that's not really what i'd be after because that impacts all objects.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
-
I know this is a very special case but still i ran headlong into it. The easiest way to implement a B+ tree on disk is using a memory mapped file. I think this is what SQL Server does, but don't quote me. However, the only way you can access memory mapped files in C# is through .NET interop which makes it useless. Because one of the points of a memory mapped file is that you can do memory allocations that are backed by disk. There's no way in hell .NET can give you that in its current incarnation, even if one were to write a custom host, because of the way a GC system works. What I'd like var foo = new int[1000000]; // backed by disk, paged automatically What I'd have to do. somepointer = VirtualAlloc(...) Write(somepointer, data) etc etc basically it works like file i/o which defeats essentially the whole purpose. =(
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
honey the codewitch wrote:
the only way you can access memory mapped files in C# is through .NET interop
Really? :-D Memory-Mapped Files | Microsoft Docs[^] MemoryMappedFile Class (System.IO.MemoryMappedFiles) | Microsoft Docs[^] (Added in .NET 4.0)
"These people looked deep within my soul and assigned me a number based on the order in which I joined." - Homer
-
honey the codewitch wrote:
the only way you can access memory mapped files in C# is through .NET interop
Really? :-D Memory-Mapped Files | Microsoft Docs[^] MemoryMappedFile Class (System.IO.MemoryMappedFiles) | Microsoft Docs[^] (Added in .NET 4.0)
"These people looked deep within my soul and assigned me a number based on the order in which I joined." - Homer
The MMF class is just a wrapper around the Interop code.
".45 ACP - because shooting twice is just silly" - JSOP, 2010
-----
You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010
-----
When you pry the gun from my cold dead hands, be careful - the barrel will be very hot. - JSOP, 2013 -
The MMF class is just a wrapper around the Interop code.
".45 ACP - because shooting twice is just silly" - JSOP, 2010
-----
You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010
-----
When you pry the gun from my cold dead hands, be careful - the barrel will be very hot. - JSOP, 2013As is quite a lot of the BCL. :)
"These people looked deep within my soul and assigned me a number based on the order in which I joined." - Homer
-
honey the codewitch wrote:
the only way you can access memory mapped files in C# is through .NET interop
Really? :-D Memory-Mapped Files | Microsoft Docs[^] MemoryMappedFile Class (System.IO.MemoryMappedFiles) | Microsoft Docs[^] (Added in .NET 4.0)
"These people looked deep within my soul and assigned me a number based on the order in which I joined." - Homer
yeah i forgot about it, but it doesn't change the same underlying issue, and isn't useful to me. It's the same thing as the wrapper i wrote a decade or more or so ago.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
-
C++/CLI
#SupportHeForShe Government can give you nothing but what it takes from somebody else. A government big enough to give you everything you want is big enough to take everything you've got, including your freedom.-Ezra Taft Benson You must accept 1 of 2 basic premises: Either we are alone in the universe or we are not alone. Either way, the implications are staggering!-Wernher von Braun
That Ezra Taft Benson, while true, is one of the dumbest remarks I've seen, and is used by some groups to put down government. The point of government is to take your money and do what's best for the people as a whole. You can argue that isn't what happens in some cases, but that misses the point. Government generally is the only entity that takes risks when the reward isn't obvious. That where just about all of our technology comes from because industry won't invest without the promise of fairly quick rewards.
-
That Ezra Taft Benson, while true, is one of the dumbest remarks I've seen, and is used by some groups to put down government. The point of government is to take your money and do what's best for the people as a whole. You can argue that isn't what happens in some cases, but that misses the point. Government generally is the only entity that takes risks when the reward isn't obvious. That where just about all of our technology comes from because industry won't invest without the promise of fairly quick rewards.
I could write pages on this but I fear aside from not being read, it would wax far too political for the lounge. I was anarchist for 26 years. I didn't so much leave it behind as I evolved, just so you have some idea of where I'm coming from. I've never particularly been a fan of states. That having been said, you're essentially right, with the following caveats: Governments are complex adaptive systems. In lay terms, they consist of so many people that they take on a life of their own. The actions of the whole are not necessarily reflective of its individual agents. The problem with this is such systems either grow or they wither and die. Stasis is rare. And they defend themselves against attempts to undermine them. Positive** prospects like limiting government do not limit government, they create defacto private government with state power and no accountability. ** i'm not using the term like "good" - i'm using it as in active vs. passive That gets dangerous and ugly when the continuation of the system is more important to the system than acting out the will of the people it's supposed to represent. And all of them fall down under the right or rather, wrong conditions. No system can perfectly contain the potential for revolt. So with those limitations in mind, I think it's a good idea to be wary of government, but I think most people regard it as necessary because anarchism doesn't scale. :)
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
-
I know this is a very special case but still i ran headlong into it. The easiest way to implement a B+ tree on disk is using a memory mapped file. I think this is what SQL Server does, but don't quote me. However, the only way you can access memory mapped files in C# is through .NET interop which makes it useless. Because one of the points of a memory mapped file is that you can do memory allocations that are backed by disk. There's no way in hell .NET can give you that in its current incarnation, even if one were to write a custom host, because of the way a GC system works. What I'd like var foo = new int[1000000]; // backed by disk, paged automatically What I'd have to do. somepointer = VirtualAlloc(...) Write(somepointer, data) etc etc basically it works like file i/o which defeats essentially the whole purpose. =(
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
-
That Ezra Taft Benson, while true, is one of the dumbest remarks I've seen, and is used by some groups to put down government. The point of government is to take your money and do what's best for the people as a whole. You can argue that isn't what happens in some cases, but that misses the point. Government generally is the only entity that takes risks when the reward isn't obvious. That where just about all of our technology comes from because industry won't invest without the promise of fairly quick rewards.
Dewey wrote:
The point of government is to take your money and do what's best for the people as a whole.
That's certainly not the point of the U.S. Federal gov't. The constitution makes it very clear that the Federal gov't is supposed to protect the people from foreign enemies, and not do much else that gets in the way of individual freedom and liberty. Of course certain forces have corrupted this to great extent.
The difficult we do right away... ...the impossible takes slightly longer.
-
I know this is a very special case but still i ran headlong into it. The easiest way to implement a B+ tree on disk is using a memory mapped file. I think this is what SQL Server does, but don't quote me. However, the only way you can access memory mapped files in C# is through .NET interop which makes it useless. Because one of the points of a memory mapped file is that you can do memory allocations that are backed by disk. There's no way in hell .NET can give you that in its current incarnation, even if one were to write a custom host, because of the way a GC system works. What I'd like var foo = new int[1000000]; // backed by disk, paged automatically What I'd have to do. somepointer = VirtualAlloc(...) Write(somepointer, data) etc etc basically it works like file i/o which defeats essentially the whole purpose. =(
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
-
Ok, This is the Lounge. THIS post here is purely programmer speak so ... take it to a forum. Pick one.
my posts of this nature were already weighed in on by Chris. they've been ruled okay for the lounge, so if you don't like them skip them or block me.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
-
i know that, but it's not efficient and doesn't solve the basic issue, which is persistent, rapidly searchable storage.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
If you want something to be truly rapid, you'd perhaps need to look at a native programming language like C++. Yes, .NET is supposed to make things easier, but not necessarily faster executing.
-
If you want something to be truly rapid, you'd perhaps need to look at a native programming language like C++. Yes, .NET is supposed to make things easier, but not necessarily faster executing.
I'm not looking for something that's bit level optimized. I'm fine with looking at complexity as for example O(log N), and that's fine, without worrying about the .NET overhead on top of the base functionality. Nah, mem mapped files, if i could use them like they were intended but under .NET, would make my programming task easier. Not necessarily make the program faster.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
-
I'm not looking for something that's bit level optimized. I'm fine with looking at complexity as for example O(log N), and that's fine, without worrying about the .NET overhead on top of the base functionality. Nah, mem mapped files, if i could use them like they were intended but under .NET, would make my programming task easier. Not necessarily make the program faster.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
honey the codewitch wrote:
I'm fine with looking at complexity as for example O(log N), and that's fine, without worrying about the .NET overhead on top of the base functionality.
Doing a full u-turn, are we? This was your response to another person on this thread:
honey the codewitch wrote:
but it's not efficient and doesn't solve the basic issue, which is persistent, rapidly searchable storage.
-
Yes, though it was added to .NET after my initial attempt at using mem mapped files from C#. Besides all that is is a wrapper like the one i had written years ago. It doesn't change the basic problem which is: var foo = new int[1000000]; //backed by disk, paged automatically, in C/C++ it's mainly because you can't use pointers in C#, and even if you use unsafe, you cannot pin objects to specific addresses in memory
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
honey the codewitch wrote:
it's mainly because you can't use pointers in C#, and even if you use unsafe, you cannot pin objects to specific addresses in memory
Yes, and that's by design.
-
I know this is a very special case but still i ran headlong into it. The easiest way to implement a B+ tree on disk is using a memory mapped file. I think this is what SQL Server does, but don't quote me. However, the only way you can access memory mapped files in C# is through .NET interop which makes it useless. Because one of the points of a memory mapped file is that you can do memory allocations that are backed by disk. There's no way in hell .NET can give you that in its current incarnation, even if one were to write a custom host, because of the way a GC system works. What I'd like var foo = new int[1000000]; // backed by disk, paged automatically What I'd have to do. somepointer = VirtualAlloc(...) Write(somepointer, data) etc etc basically it works like file i/o which defeats essentially the whole purpose. =(
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
Environments designated to making simple things simple tend to make complicated things more complicated. Or maybe not, in the case of .NET, there's the unsafe keyword which allows you to work on raw pointers. However, that indeed doesn't help much with disk stuff. See it this way, C# is the polar opposite of C here. In C, complex things and simple things work about the same way at the cost of simple things being ridiculously complicated. I remember a project of mine where I combined the strengths of two worlds, I did my low-level in C (technically C++, but I had raw pointers at work), exposed a C interface and did my business logic in C#. Such a mixed design is of course too easy to get wrong, but I think I've managed to make it make sense.
-
Environments designated to making simple things simple tend to make complicated things more complicated. Or maybe not, in the case of .NET, there's the unsafe keyword which allows you to work on raw pointers. However, that indeed doesn't help much with disk stuff. See it this way, C# is the polar opposite of C here. In C, complex things and simple things work about the same way at the cost of simple things being ridiculously complicated. I remember a project of mine where I combined the strengths of two worlds, I did my low-level in C (technically C++, but I had raw pointers at work), exposed a C interface and did my business logic in C#. Such a mixed design is of course too easy to get wrong, but I think I've managed to make it make sense.
I completely agree, and I've done similar. It's just that here, it's not really my goal to make something tied to the windows platform specifically, and pretty much no matter the route i take as soon as I'm relying on that mixed managed and unmanaged code I'm also inherently tying it to a platform. Sure I can cross compile the unmanaged code, but even then, it's one package per platform - not what i want.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
