Data sharing / combining data of multiple SBS-1 boxes

This forum is for users who are developing applications (or resources) to work alongside BaseStation (i.e. add-ons) and who wish to discuss development techniques.

Moderators: KineticChris, KineticDavid, Moderators

roger_tinembart
Posts: 19
Joined: February 24th, 2010, 2:53 pm
Location: Giswil, Swizerland

Data sharing / combining data of multiple SBS-1 boxes

Post by roger_tinembart » February 24th, 2010, 4:03 pm

Dear group

I am planning to start a project to combine the data of multiple SBS-1 boxes. The goal is to collect the data from as lot of SBS-1 boxes as possible, combine the data on a central server and make the combined data accessible to all the participants. So everyone that sends data may also receive data. All that with only a short delay (planned 10 - 15 seconds roundtrip) so that the data is still accurate when it arrives back at you.

Between the participating computer(s) and the central server, a proprietary format with high compression will be used. The combined data will be offered on a local port of every participating computer in the same format like port 30003 of the Basestation.

You would have to install a small windows application which gets the data of your SBS at port 30003 of the Basestation-App, compresses your data, sends it to the central server, receives the combined data, decompresses it and offers it to you in 'standard-format'. Visualization is not planned at the moment, i focus on concentrating the data. But i guess there are enough tools 'out there' which handle the visualisation part.

If this sounds interesting to anyone or if you have further questions or comments please reply here. If someone has legal concerns please reply also...

I hope the description is clear enough as i speak (swiss-)german as mother tongue.

Thanks for reading

Roger
Roger Tinembart

edgy31

Re: Data sharing / combining data of multiple SBS-1 boxes

Post by edgy31 » February 24th, 2010, 4:15 pm

What software language are you going to use?
Last edited by edgy31 on February 26th, 2010, 8:12 pm, edited 1 time in total.

User avatar
Anmer
Posts: 6663
Joined: November 22nd, 2007, 8:35 pm
Location: England
Contact:

Re: Data sharing / combining data of multiple SBS-1 boxes

Post by Anmer » February 24th, 2010, 4:17 pm

Good luck.

You may find these of interest:

Networking the SBS-1

Data Sharing Trial

The Data Sharing Trial is at an advanced stage.
Anmer
Radarspotting.com
Radarspotting since 2005

roger_tinembart
Posts: 19
Joined: February 24th, 2010, 2:53 pm
Location: Giswil, Swizerland

Re: Data sharing / combining data of multiple SBS-1 boxes

Post by roger_tinembart » February 25th, 2010, 6:00 am

edgy31 wrote:Lots of competition there, with many duplicate sharers.
Well i hope for many sharers... But that is not a problem. As each object has a unique address it is easy to merge the data.
edgy31 wrote:I tried compression myself. You can only save about 10 bytes per packet.
What kind of 'packet' are you talking about and how did you try to compress it? Based on my tests, i was able to 'compress' the data stream which i receive from port 30003 down to at about 5% of its original size, still able to 'regenerate' it quite accurate on the receiver side. In numbers: Sending the actual 'data' of 100 planes every 5 seconds in compressed format would take an average bandwith of about 22 kBits per second. With 'data' i mean the message fields 1,2, 5 - 22 of type MSG of the stream received on port 30003. The fields 3 and 4 refer to the local database and is - in my understanding - of no use for others. So this means to share 100 planes, an old 56 kBit Modem would have enough upload bandwith...
edgy31 wrote:Kind of a redundant data format.
Yes, that's true. And this is the reason why its very good compressable. I had the idea of choosing this format as it is a well known 'standard'.
edgy31 wrote:What software language are you going to use?
I am planning to use C++ for the client application.

BTW: There would be no need to open any ports on the participating clients or have a fixed IP. All the communication would be initiated from the client side.
Roger Tinembart

edgy31

Re: Data sharing / combining data of multiple SBS-1 boxes

Post by edgy31 » February 25th, 2010, 8:39 am

Good luck with your project!
Last edited by edgy31 on February 26th, 2010, 8:13 pm, edited 3 times in total.

roger_tinembart
Posts: 19
Joined: February 24th, 2010, 2:53 pm
Location: Giswil, Swizerland

Re: Data sharing / combining data of multiple SBS-1 boxes

Post by roger_tinembart » February 25th, 2010, 8:52 am

Anmer wrote:Good luck.

... snip ...

The Data Sharing Trial is at an advanced stage.
Hi anmer

Thank you very much for your information. I did not know about your thread and the MMS service it is using. A few questions:
  • From the document 'Networking_th_SBS-1' i learned that the bandwidth demand is very high. In your thread you are talking about MMS1 and MMS2-service. Are these improvements of the original standard? Was it possible to lower the bandwidth demand? I ask because to reduce bandwidth is one of my primary targets. I am calculating with a maximum of 22 kBit/s per 100 planes in '5-seconds real time', even less (bandwidth) in the future.
  • Is it still possible to join your testing-group just to see / 'feel' how it is working?
  • I am asking here because i did not want to disturb your thread or the progress of your project. Anyway: Would you mind if i post my results / further questions / comparisons also in your thread?
Roger Tinembart

User avatar
Anmer
Posts: 6663
Joined: November 22nd, 2007, 8:35 pm
Location: England
Contact:

Re: Data sharing / combining data of multiple SBS-1 boxes

Post by Anmer » February 25th, 2010, 11:48 am

Please post any comments about the Data Sharing trial in the Radarspotters thread or here if they are of general interest.

MMS2 was initially to reduce use of bandwidth and whilst this has been achieved, there is further work to be done. There are also functional enhancements.

There's a waiting list of "volunteers" and at the moment we aren't extending the group as we have further enhancements to test. However, I will advise the developer of your specific interest.
Anmer
Radarspotting.com
Radarspotting since 2005

User avatar
Halcones
Posts: 861
Joined: March 16th, 2007, 6:06 pm
Location: Malmesbury, Wiltshire

Re: Data sharing / combining data of multiple SBS-1 boxes

Post by Halcones » February 25th, 2010, 10:04 pm

Done it

+ LINUX
+ perl
+ MySQL

After that - anything is possible

I had 4 SBS-1's runninng at once into a central server

+ feeding aprs
+ feeding google earth
+ feeding telnet output back again
Kinetic Beta team member

Image

edgy31

Re: Data sharing / combining data of multiple SBS-1 boxes

Post by edgy31 » February 26th, 2010, 8:16 pm

Another thread that goes nowhere...

roger_tinembart
Posts: 19
Joined: February 24th, 2010, 2:53 pm
Location: Giswil, Swizerland

Re: Data sharing / combining data of multiple SBS-1 boxes

Post by roger_tinembart » February 27th, 2010, 11:41 am

vk4tec wrote:Done it

+ LINUX
+ perl
+ MySQL

After that - anything is possible

---snip---
Thanks also for the input concerning linux, perl and mysql. Possibly this is useful in a later stage when the data fusioning part is working. But i think to reach my primary target which is combining the data of as much as possible ADSB-receivers as possible with a small demand of bandwidth and 'as real time as possible', i can't use this approach now. I want to focus on windows platform as the Basestation application is also working on windows, and i decided to do it as a native application - no java vm or DatNET required.
Roger Tinembart

roger_tinembart
Posts: 19
Joined: February 24th, 2010, 2:53 pm
Location: Giswil, Swizerland

Re: Data sharing / combining data of multiple SBS-1 boxes

Post by roger_tinembart » February 27th, 2010, 11:51 am

edgy31 wrote:Another thread that goes nowhere...
I can't tell you where that thread is going to right now. What i know is:
  • i started with the development of the client application this week, the server application will follow…
  • the first test of the application (client and server) will start in April
  • the application will work great :D
  • i will post information concerning the progress of the project here
  • i would be happy if i could find some people here who are interested in joining the test
Thanks for reading and feedback
vk4tec wrote:So delete it and recover the space
hä???
Roger Tinembart

10032050
Posts: 143
Joined: June 28th, 2007, 10:13 pm
Location: Earth

Re: Data sharing / combining data of multiple SBS-1 boxes

Post by 10032050 » April 2nd, 2010, 12:20 pm

hey all I found this www site http://www.flightradar24.com/?lat=51d87 ... 625&zoom=8

its also great feed for SBS users I am now feeding the Southern Area of U.K.

what do you think

cheers

roger_tinembart
Posts: 19
Joined: February 24th, 2010, 2:53 pm
Location: Giswil, Swizerland

Re: Data sharing / combining data of multiple SBS-1 boxes

Post by roger_tinembart » April 2nd, 2010, 12:48 pm

10032050 wrote:hey all I found this www site http://www.flightradar24.com

its also great feed for SBS users I am now feeding the Southern Area of U.K.

what do you think

cheers
Wow, what a great website! I just knew this site: http://www.radarvirtuel.com/ which shows around 1000 planes all around europe, but their positions have a delay of up to 4 minutes. And this delay was also the reason for me to start my project. On http://www.flightradar24.com it seams to be 'realtime'. Great!

Thanks you for this link

Edit: http://www.radarvirtuel.com seams to have boosted thier performance. Now (June 2010) the data seams to be 'realtime' also on this site.
Last edited by roger_tinembart on June 19th, 2010, 6:21 am, edited 1 time in total.

roger_tinembart
Posts: 19
Joined: February 24th, 2010, 2:53 pm
Location: Giswil, Swizerland

Re: Data sharing / combining data of multiple SBS-1 boxes

Post by roger_tinembart » April 19th, 2010, 7:00 am

Dear group

The developpment of the first version of my application is almost completed. At the moment, i am running some inhouse-tests with five simulators which are replaying recorded data to get a feeling for the performance and to check for memory-problems. It's looking very good for now...

The current version is able to receive data from a local connected BaseStation, send the data to a server, receive the merged data from the server and offer it in SBS-socket format (i.e. in the same format as if you would connect to BaseStation). So any software that understands the SBS-socket format should be able to understand the format of my application. Small limitation: The fields 'SessionId', 'AircraftId' and 'FlightID' are not transmitted at the moment.

If there are any volonteers out there who are willing to install the application (just copy and run) please contact me. I will send instructions on how to download and install it. The public test will start at 1.May.

Thanks, Roger
Roger Tinembart

User avatar
westrowops
Posts: 205
Joined: August 18th, 2006, 4:33 pm
Contact:

Re: Data sharing / combining data of multiple SBS-1 boxes

Post by westrowops » June 9th, 2010, 11:53 pm

ALready been done, its called Planeplotter !! why re-invent the wheel?
I used to have an open mind, but my brains kept falling out

Post Reply