File Sharing App (DropBox) Propose Architecture
-
Here's some thoughts on a potential design for a file transfer application. I welcome any input you may have: Purpose Provide a real-time file transfer service that allows users to upload/download files to/from a server and to provide automatic synchronization between the server and the user machines. A user-specified folder structure can be defined on the local machine. When a file or folder is added, modified, or deleted (Known as a Change) in any file or folder in this structure is the added, modified, or deleted on the server accordingly. When a Change occurs, all users must be notified, and automatic folder/file synchronization between the server and the local file structures must be automatic and transparent to the users. Proposed Solution (Prototype) - The local machine will run a FileSystemWatcher hosted by a windows service. - FTP wil be used to transfer files to/from the server. - A SignalR service will be hosted on the server and function as the mechanism for maintainin connection to and communicating Changes Messages between clients. A Change Message contains the following data: 1. Client Id (GUID) - The ID of the client originating the change 2. Item Type - File or Folder 2. Name - Full path and name of the file or folder 4. Action - Create, Modify, Delete 5. Location - Client or Server Use Case 1 - File Added A user drags a file into a folder called c:\TheApp\SomeFolder\MyFile.txt. The file does not already exist in the folder. The FileSystemWatcher detects the new file, FTP's it to the server with progress reporing. Once the upload is commplete then the client transmists a message to the server as such: Client: {6FD41E1C-0057-44E4-B1AA-E0A4A263ABA3} ItemType: File Name: "c:\TheApp\SomeFolder\MyFile.txt" Action New Location: Client The server recieves the message, verifys that the file exists on the server, then generates and sends the following message to all clients except the sender: Client: {6FD41E1C-0057-44E4-B1AA-E0A4A263ABA3} ItemType: File Name: "SomeFolder\MyFile.txt" Action New Location: Server The client recieves the message and then initiates an FTP of the file "SomeFolder\MyFile.txt" to "c:\TheApp\SomeFolder\MyFile.txt". Use Case 2 - Folder Deleted A user removes the folder c:\TheApp\SomeFolder\. The FileSystemWatcher detects the change and transmists a message to the server as such: Client: {6FD41E1C-0057-44E4-B1AA-E0A4A263ABA3} ItemType: Folder Name: "c:\TheApp\SomeFolde
-
Here's some thoughts on a potential design for a file transfer application. I welcome any input you may have: Purpose Provide a real-time file transfer service that allows users to upload/download files to/from a server and to provide automatic synchronization between the server and the user machines. A user-specified folder structure can be defined on the local machine. When a file or folder is added, modified, or deleted (Known as a Change) in any file or folder in this structure is the added, modified, or deleted on the server accordingly. When a Change occurs, all users must be notified, and automatic folder/file synchronization between the server and the local file structures must be automatic and transparent to the users. Proposed Solution (Prototype) - The local machine will run a FileSystemWatcher hosted by a windows service. - FTP wil be used to transfer files to/from the server. - A SignalR service will be hosted on the server and function as the mechanism for maintainin connection to and communicating Changes Messages between clients. A Change Message contains the following data: 1. Client Id (GUID) - The ID of the client originating the change 2. Item Type - File or Folder 2. Name - Full path and name of the file or folder 4. Action - Create, Modify, Delete 5. Location - Client or Server Use Case 1 - File Added A user drags a file into a folder called c:\TheApp\SomeFolder\MyFile.txt. The file does not already exist in the folder. The FileSystemWatcher detects the new file, FTP's it to the server with progress reporing. Once the upload is commplete then the client transmists a message to the server as such: Client: {6FD41E1C-0057-44E4-B1AA-E0A4A263ABA3} ItemType: File Name: "c:\TheApp\SomeFolder\MyFile.txt" Action New Location: Client The server recieves the message, verifys that the file exists on the server, then generates and sends the following message to all clients except the sender: Client: {6FD41E1C-0057-44E4-B1AA-E0A4A263ABA3} ItemType: File Name: "SomeFolder\MyFile.txt" Action New Location: Server The client recieves the message and then initiates an FTP of the file "SomeFolder\MyFile.txt" to "c:\TheApp\SomeFolder\MyFile.txt". Use Case 2 - Folder Deleted A user removes the folder c:\TheApp\SomeFolder\. The FileSystemWatcher detects the change and transmists a message to the server as such: Client: {6FD41E1C-0057-44E4-B1AA-E0A4A263ABA3} ItemType: Folder Name: "c:\TheApp\SomeFolde
-
What happens if a user is not logged on? Do they get "synced" when they log on? How "current" does the syncing have to be? Why? Design choices are premature at this point; IMO.
Gerry Schmitz wrote:
What happens if a user is not logged on? Do they get "synced" when they log on?
Yes
Gerry Schmitz wrote:
How "current" does the syncing have to be? Why?
Real-time. This app wil be applying business rules to documents placed in the folder structure, so it's important to keep the folders/files as up to date as possible. For example, there may be a requirement to destroy a doc automatically after a certain date/time. The server would send a Destroy message to the clients, and then doc would be removed.
Gerry Schmitz wrote:
Design choices are premature at this point; IMO
Not sure I agree. We've been discussing and documenting the requirements for a year now. It's time to prototype, so I'm looking for technologies that will fulfil the requirements and then to get started. Thanks for your input.
If it's not broken, fix it until it is
-
Gerry Schmitz wrote:
What happens if a user is not logged on? Do they get "synced" when they log on?
Yes
Gerry Schmitz wrote:
How "current" does the syncing have to be? Why?
Real-time. This app wil be applying business rules to documents placed in the folder structure, so it's important to keep the folders/files as up to date as possible. For example, there may be a requirement to destroy a doc automatically after a certain date/time. The server would send a Destroy message to the clients, and then doc would be removed.
Gerry Schmitz wrote:
Design choices are premature at this point; IMO
Not sure I agree. We've been discussing and documenting the requirements for a year now. It's time to prototype, so I'm looking for technologies that will fulfil the requirements and then to get started. Thanks for your input.
If it's not broken, fix it until it is
How can a client be "real-time" when it is off-line (and in effect acting like a "device")? I see nothing in your descriptions that requires a "real-time" solution. All "to do" items can be logged in a database and dispatched based on triggers, async callbacks and / or scheduled take up processing.
-
How can a client be "real-time" when it is off-line (and in effect acting like a "device")? I see nothing in your descriptions that requires a "real-time" solution. All "to do" items can be logged in a database and dispatched based on triggers, async callbacks and / or scheduled take up processing.
Gerry Schmitz wrote:
How can a client be "real-time" when it is off-line (and in effect acting like a "device")?
The app is never off line. Since SignalR can maintain a connection to the client via WebSockets, then the app is able to communicate between the client an server at all times.
Gerry Schmitz wrote:
I see nothing in your descriptions that requires a "real-time" solution.
You're right in that we probably could accomplish our objectives by polling the server at specified intervals. But given the ability of SignalR [^]there's no reason not to.
If it's not broken, fix it until it is
-
Gerry Schmitz wrote:
How can a client be "real-time" when it is off-line (and in effect acting like a "device")?
The app is never off line. Since SignalR can maintain a connection to the client via WebSockets, then the app is able to communicate between the client an server at all times.
Gerry Schmitz wrote:
I see nothing in your descriptions that requires a "real-time" solution.
You're right in that we probably could accomplish our objectives by polling the server at specified intervals. But given the ability of SignalR [^]there's no reason not to.
If it's not broken, fix it until it is
-
Well time will tell. We'll know for sure once the prototyping is done. Once it's done I'll post it here as an article so I can get more feedback. Thanks for your input.
If it's not broken, fix it until it is