Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • World
  • Users
  • Groups
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Code Project
  1. Home
  2. The Lounge
  3. I thought .NET was supposed to make things easier, if anything, than unmanaged code.

I thought .NET was supposed to make things easier, if anything, than unmanaged code.

Scheduled Pinned Locked Moved The Lounge
csharpdatabasesql-servercomsysadmin
111 Posts 19 Posters 5 Views 1 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • Z zezba9000

    Use unsafe code with pointers just as you would in C and you can do the same thing as you could in C.

    H Offline
    H Offline
    honey the codewitch
    wrote on last edited by
    #72

    Nope, because you can't pin an object to a specific address in memory. Ergo I can't back a .NET instance with a memory mapped file. Memory mapped files work by mapping a portion of the process address space to your application. So you can read and write the file doing pointer ops. However, in .NET you can only get pointers to the GC heap by pinning objects. So I cannot do

    var arr = new int[100000000];// backed by memory mapped file

    in any variation. Even if I were to pin the array to get the pointer, the pointer wouldn't be part of that mapped address space. At best, I have to *copy* the entire object into that mapped region, which absolutely defeats the whole point of why I'd use mem mapped I/O here in the first place. There's a half measure using Span, but it only works in Standard and Core, not the DNF

    When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.

    Z 1 Reply Last reply
    0
    • H honey the codewitch

      Nope, because you can't pin an object to a specific address in memory. Ergo I can't back a .NET instance with a memory mapped file. Memory mapped files work by mapping a portion of the process address space to your application. So you can read and write the file doing pointer ops. However, in .NET you can only get pointers to the GC heap by pinning objects. So I cannot do

      var arr = new int[100000000];// backed by memory mapped file

      in any variation. Even if I were to pin the array to get the pointer, the pointer wouldn't be part of that mapped address space. At best, I have to *copy* the entire object into that mapped region, which absolutely defeats the whole point of why I'd use mem mapped I/O here in the first place. There's a half measure using Span, but it only works in Standard and Core, not the DNF

      When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.

      Z Offline
      Z Offline
      zezba9000
      wrote on last edited by
      #73

      This is what "Span" is for. Return that instead of an array and your API will be "managed" in the sense to an external user. Or you can use "GCHandle.Alloc" to pin a managed array in memory. GCHandle.Alloc Method (System.Runtime.InteropServices) | Microsoft Docs[^] Using Span is a better idea though.

      H 1 Reply Last reply
      0
      • Z zezba9000

        This is what "Span" is for. Return that instead of an array and your API will be "managed" in the sense to an external user. Or you can use "GCHandle.Alloc" to pin a managed array in memory. GCHandle.Alloc Method (System.Runtime.InteropServices) | Microsoft Docs[^] Using Span is a better idea though.

        H Offline
        H Offline
        honey the codewitch
        wrote on last edited by
        #74

        Another commenter pointed out Span, which I had forgotten about. It's a half measure** but gets me closer, however, it does not work in the DNF, only in Core and Standard i think. GCHandle.Alloc will not let me pin to a specific address. In order for .NET to do anything beyond span it would have to allow you to create unmanaged heaps to allocate objects on which the collector wouldn't collect. It's designed such that it can't work, and I accept that. It would just be nice if it had some kind of feature like the above to enable it. ** I don't actually want to store an array, i was using that as an example. My datastructure is complex and nested, which is precisely why i'd like to use vmem

        When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.

        Z 1 Reply Last reply
        0
        • Z zezba9000

          Well in that case for .NET Framework unsafe code is the only answer. Or just return a IntPtr and let the user decide if they want to marshal or use unsafe code if this is for an API. Thats how MS does it.

          H Offline
          H Offline
          honey the codewitch
          wrote on last edited by
          #75

          Not applicable for this.

          When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.

          Z 1 Reply Last reply
          0
          • H honey the codewitch

            Another commenter pointed out Span, which I had forgotten about. It's a half measure** but gets me closer, however, it does not work in the DNF, only in Core and Standard i think. GCHandle.Alloc will not let me pin to a specific address. In order for .NET to do anything beyond span it would have to allow you to create unmanaged heaps to allocate objects on which the collector wouldn't collect. It's designed such that it can't work, and I accept that. It would just be nice if it had some kind of feature like the above to enable it. ** I don't actually want to store an array, i was using that as an example. My datastructure is complex and nested, which is precisely why i'd like to use vmem

            When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.

            Z Offline
            Z Offline
            zezba9000
            wrote on last edited by
            #76

            Well in that case for .NET Framework unsafe code is the only answer. Or just return a IntPtr and let the user decide if they want to marshal or use unsafe code if this is for an API. Thats how MS does it.

            H 1 Reply Last reply
            0
            • H honey the codewitch

              Not applicable for this.

              When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.

              Z Offline
              Z Offline
              zezba9000
              wrote on last edited by
              #77

              MS uses unsafe code all over the place in their own frameworks. Not sure how its an issue but ok. Even if you could address managed array to a native buffer that would make it unsafe usage.

              H 1 Reply Last reply
              0
              • Z zezba9000

                MS uses unsafe code all over the place in their own frameworks. Not sure how its an issue but ok. Even if you could address managed array to a native buffer that would make it unsafe usage.

                H Offline
                H Offline
                honey the codewitch
                wrote on last edited by
                #78

                unsafe isn't a problem. This has nothing to do with a desire not to run unsafe code Again, what it has to do with is mapping a complex managed data structure to a file using Vmem. It's not doable. At best, you resolve to something much like standard file i/o which defeats the whole reason i'd use vmem. Adding, span would help here, but it doesn't go near enough to to be a solution since these data structures aren't simple arrays.

                When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.

                Z 1 Reply Last reply
                0
                • H honey the codewitch

                  unsafe isn't a problem. This has nothing to do with a desire not to run unsafe code Again, what it has to do with is mapping a complex managed data structure to a file using Vmem. It's not doable. At best, you resolve to something much like standard file i/o which defeats the whole reason i'd use vmem. Adding, span would help here, but it doesn't go near enough to to be a solution since these data structures aren't simple arrays.

                  When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.

                  Z Offline
                  Z Offline
                  zezba9000
                  wrote on last edited by
                  #79

                  Then just wrap the C API that does what you want. Anything you can do in C you can do in C# by using it like C. So it is doable just maybe not how you like.

                  H 1 Reply Last reply
                  0
                  • Z zezba9000

                    Then just wrap the C API that does what you want. Anything you can do in C you can do in C# by using it like C. So it is doable just maybe not how you like.

                    H Offline
                    H Offline
                    honey the codewitch
                    wrote on last edited by
                    #80

                    can't do that either because it ties it to a platform. the goal is cross platform. At best i'd have to do two binaries per platform. and it would only support a limited number of them. So it doesn't fulfill the requirements. If I didn't care about being cross platform, I'd just use mixed mode C++. It's okay that this isn't doable. I accept it. It would just be nice if it was.

                    When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.

                    Z 1 Reply Last reply
                    0
                    • D Dewey

                      That Ezra Taft Benson, while true, is one of the dumbest remarks I've seen, and is used by some groups to put down government. The point of government is to take your money and do what's best for the people as a whole. You can argue that isn't what happens in some cases, but that misses the point. Government generally is the only entity that takes risks when the reward isn't obvious. That where just about all of our technology comes from because industry won't invest without the promise of fairly quick rewards.

                      S Offline
                      S Offline
                      Steven1218
                      wrote on last edited by
                      #81

                      I have seen a similar quote attributed to Thomas Jefferson. I think it is less about government per se, but more about the scale of government.

                      1 Reply Last reply
                      0
                      • H honey the codewitch

                        can't do that either because it ties it to a platform. the goal is cross platform. At best i'd have to do two binaries per platform. and it would only support a limited number of them. So it doesn't fulfill the requirements. If I didn't care about being cross platform, I'd just use mixed mode C++. It's okay that this isn't doable. I accept it. It would just be nice if it was.

                        When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.

                        Z Offline
                        Z Offline
                        zezba9000
                        wrote on last edited by
                        #82

                        You don't need multiple native or managed binaries for this. You can just wrap each platform and detect what platform you're on in the .NET runtime via RuntimeInformation class. So you would probably only need one for Win32 and one for POSIX (aka Linux, BSD and macOS). Because a .NET lib doesn't exist for what you need thats portable, you will have to make one that is.

                        H 1 Reply Last reply
                        0
                        • Z zezba9000

                          You don't need multiple native or managed binaries for this. You can just wrap each platform and detect what platform you're on in the .NET runtime via RuntimeInformation class. So you would probably only need one for Win32 and one for POSIX (aka Linux, BSD and macOS). Because a .NET lib doesn't exist for what you need thats portable, you will have to make one that is.

                          H Offline
                          H Offline
                          honey the codewitch
                          wrote on last edited by
                          #83

                          Once again, I cannot map objects to a specific process address space in .NET. No amount of marshalling or unsafe code will change that. It might be possible to create an unmanaged cross compiled library and link to the different calling conventions etc, by conditional P/Invoke, at the cost of extra complexity and adds marshalling to a class that shouldn't have it for perf reasons, and still requiring a cross compiled unmanaged binary and still therefore being limited to certain platforms, not all .NET platforms. So again, it doesn't fulfill the basic requirement because of that latter bit. I accept that it can't be done in .NET

                          When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.

                          Z 1 Reply Last reply
                          0
                          • H honey the codewitch

                            Once again, I cannot map objects to a specific process address space in .NET. No amount of marshalling or unsafe code will change that. It might be possible to create an unmanaged cross compiled library and link to the different calling conventions etc, by conditional P/Invoke, at the cost of extra complexity and adds marshalling to a class that shouldn't have it for perf reasons, and still requiring a cross compiled unmanaged binary and still therefore being limited to certain platforms, not all .NET platforms. So again, it doesn't fulfill the basic requirement because of that latter bit. I accept that it can't be done in .NET

                            When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.

                            Z Offline
                            Z Offline
                            zezba9000
                            wrote on last edited by
                            #84

                            You said using unsafe code wasn't an issue? So again anything you can do in C can be done in C# if you use unsafe code if thats not an issue to you. It sounds like you don't fully understand what you can do in unsafe code in C#. Cross compiling a native binary would be the same as just using unsafe code in C# and wraping the C methods you need.

                            H 1 Reply Last reply
                            0
                            • Z zezba9000

                              You said using unsafe code wasn't an issue? So again anything you can do in C can be done in C# if you use unsafe code if thats not an issue to you. It sounds like you don't fully understand what you can do in unsafe code in C#. Cross compiling a native binary would be the same as just using unsafe code in C# and wraping the C methods you need.

                              H Offline
                              H Offline
                              honey the codewitch
                              wrote on last edited by
                              #85

                              zezba9000 wrote:

                              So again anything you can do in C can be done in C# if you use unsafe code

                              This is categorically incorrect. There is no way to map a .NET object to a file backed portion of the process address space and there likely never will be in .NET. Copying the object to that space or otherwise serializing it is not the same thing at all. I'm going attempt to summarize: You cannot create an object at a specific memory location in managed code.

                              When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.

                              Z 1 Reply Last reply
                              0
                              • H honey the codewitch

                                zezba9000 wrote:

                                So again anything you can do in C can be done in C# if you use unsafe code

                                This is categorically incorrect. There is no way to map a .NET object to a file backed portion of the process address space and there likely never will be in .NET. Copying the object to that space or otherwise serializing it is not the same thing at all. I'm going attempt to summarize: You cannot create an object at a specific memory location in managed code.

                                When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.

                                Z Offline
                                Z Offline
                                zezba9000
                                wrote on last edited by
                                #86

                                You don't need to if you use pointers in C# just like you would in C. AKA "unsafe code". Unsafe code doesn't require the use of managed objects. Use int* instead of int[] then it will work. More info: Unsafe code and pointers - C# Programming Guide | Microsoft Docs[^]

                                H 1 Reply Last reply
                                0
                                • Z zezba9000

                                  You don't need to if you use pointers in C# just like you would in C. AKA "unsafe code". Unsafe code doesn't require the use of managed objects. Use int* instead of int[] then it will work. More info: Unsafe code and pointers - C# Programming Guide | Microsoft Docs[^]

                                  H Offline
                                  H Offline
                                  honey the codewitch
                                  wrote on last edited by
                                  #87

                                  I'm sorry but yes you do if you want that region to be backed by a memory mapped file. otherwise you're just copying/serializing objects into that space at best. to back an object with a memory mapped region of the process address space you'd have to be able to pin an object to a specific address - not really possible in a garbage collected system without some sort of significant work around.

                                  When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.

                                  Z 1 Reply Last reply
                                  0
                                  • H honey the codewitch

                                    I'm sorry but yes you do if you want that region to be backed by a memory mapped file. otherwise you're just copying/serializing objects into that space at best. to back an object with a memory mapped region of the process address space you'd have to be able to pin an object to a specific address - not really possible in a garbage collected system without some sort of significant work around.

                                    When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.

                                    Z Offline
                                    Z Offline
                                    zezba9000
                                    wrote on last edited by
                                    #88

                                    No you don't have to copy memory if you use a native pointer to a memory mapped file. In C# a pointer like "int*" is the same as a pointer in C/C++. There is no difference. That pointer in C# will directly be pointing to your native memory buffer you allocated from a native Win32 or POSIX API. Pointers in C# can point to literally anything just as they can in C/C++. That includes managed and unmanaged memory locations. For example this psuedo C# code would work:

                                    [DllImport("nativeLib")]
                                    static unsafe extern int* CreateNativeMemoryMappedFile();

                                    unsafe void WriteMemoryMappedFile()
                                    {
                                    int* nativeMemory = CreateNativeMemoryMappedFile();
                                    nativeMemory[0] = 123;// this is writing directly from the native buffer (not a copy)
                                    if (nativeMemory[1] == 321) return;// this is reading directly from the native buffer (not a copy)
                                    }

                                    H 1 Reply Last reply
                                    0
                                    • Z zezba9000

                                      No you don't have to copy memory if you use a native pointer to a memory mapped file. In C# a pointer like "int*" is the same as a pointer in C/C++. There is no difference. That pointer in C# will directly be pointing to your native memory buffer you allocated from a native Win32 or POSIX API. Pointers in C# can point to literally anything just as they can in C/C++. That includes managed and unmanaged memory locations. For example this psuedo C# code would work:

                                      [DllImport("nativeLib")]
                                      static unsafe extern int* CreateNativeMemoryMappedFile();

                                      unsafe void WriteMemoryMappedFile()
                                      {
                                      int* nativeMemory = CreateNativeMemoryMappedFile();
                                      nativeMemory[0] = 123;// this is writing directly from the native buffer (not a copy)
                                      if (nativeMemory[1] == 321) return;// this is reading directly from the native buffer (not a copy)
                                      }

                                      H Offline
                                      H Offline
                                      honey the codewitch
                                      wrote on last edited by
                                      #89

                                      you cannot create objects at that pointer. ergo, no vmem backed objects. you can do pointer ops, but then you're copying object data from the object (on GC heap) to the unmanaged heap. If I do a memcpy, or pointer based assignment in a for loop, either way i'm copying the data. and doing so, even if i did, is really no different than calling (filestream).Write(...) which would be more direct. there is no way around it because of how a GC works. if you think I'm wrong show me how to *create an object* at a specific address in .NET - because that's the only way to back an object with vmem, short of creating a custom CLI host in C(++) or some other unmanaged language and AFAICT would have to back an entire appdomain with vmem for it to work.

                                      When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.

                                      Z 1 Reply Last reply
                                      0
                                      • H honey the codewitch

                                        you cannot create objects at that pointer. ergo, no vmem backed objects. you can do pointer ops, but then you're copying object data from the object (on GC heap) to the unmanaged heap. If I do a memcpy, or pointer based assignment in a for loop, either way i'm copying the data. and doing so, even if i did, is really no different than calling (filestream).Write(...) which would be more direct. there is no way around it because of how a GC works. if you think I'm wrong show me how to *create an object* at a specific address in .NET - because that's the only way to back an object with vmem, short of creating a custom CLI host in C(++) or some other unmanaged language and AFAICT would have to back an entire appdomain with vmem for it to work.

                                        When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.

                                        Z Offline
                                        Z Offline
                                        zezba9000
                                        wrote on last edited by
                                        #90

                                        "

                                        Quote:

                                        if you think I'm wrong show me how to *create an object* at a specific address in .NET

                                        Invoke the C method from C# you would use to allocation memory at that specific vmem address. Keep a reference to that address in C# as a pointer as you would in C. No difference from doing it in C. Again if you can address it in C you can address it in C# using pointers. Can you show me what method you're invoking in C that allocates memory at a specific address? Then I can give an example and it would remove the confusion. I can only guess at what methods you're trying to use.

                                        H 2 Replies Last reply
                                        0
                                        • Z zezba9000

                                          "

                                          Quote:

                                          if you think I'm wrong show me how to *create an object* at a specific address in .NET

                                          Invoke the C method from C# you would use to allocation memory at that specific vmem address. Keep a reference to that address in C# as a pointer as you would in C. No difference from doing it in C. Again if you can address it in C you can address it in C# using pointers. Can you show me what method you're invoking in C that allocates memory at a specific address? Then I can give an example and it would remove the confusion. I can only guess at what methods you're trying to use.

                                          H Offline
                                          H Offline
                                          honey the codewitch
                                          wrote on last edited by
                                          #91

                                          what C method? the method to create the object? that's not a .NET object. I can already do all this mapping in C. There's no point in any of that.

                                          When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.

                                          1 Reply Last reply
                                          0
                                          Reply
                                          • Reply as topic
                                          Log in to reply
                                          • Oldest to Newest
                                          • Newest to Oldest
                                          • Most Votes


                                          • Login

                                          • Don't have an account? Register

                                          • Login or register to search.
                                          • First post
                                            Last post
                                          0
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular
                                          • World
                                          • Users
                                          • Groups