Next Generation Data System & Collaborative Portal .fr

Nov 2, 2006 - Apply in other field programs and observatories. • Allow real-time queries/visualizations. (Metadata Oriented Query Assistant – MOQUA).
511KB taille 3 téléchargements 312 vues
Next Generation Data System & Collaborative Portal

November 2, 2006 James G Bellingham, Michael A Godin Dorota Kolber, Lionel Pawlowski Monterey Bay Aquarium Research Institute

Data/Collaboration Highlights • “Virtual Experiments” preceded the field work • Most researchers remained at home institutions. • More real-time model outputs • Standard products for comparing data • A variety of ways to share findings and data – Collaborative system allowed investigators to post analyses and/or link to own sites 2006/11/02 Bellingham & Godin

Data/Collaboration Highlights • Automated data collection system – Included data streams from outside the experiment – Often fragile with respect to source data changes – Mostly worked, but still finding some problems (thanks, Wayne!)

• Non-ASAP data originators often ‘last minute’ – Difficult to handle – Some errors in common data format resulted

• Collaborative portal used in lieu of RTOC meetings

2006/11/02 Bellingham & Godin

Data System Throughput From July 20 to September 14, 2006: • 75,406 raw files (160 GB) collected by central server. • 187,494 common format files (129 GB) created. • 38 data formats handled • 2200 GB data served

2006/11/02 Bellingham & Godin

MB2006 Data Collection 7/6

7/13

7/20

7/27

8/3

8/10

8/17

8/24

8/31

9/7

9/14

mbari_auv_dorado389 codar_mybg calpoly_remus_auv_loco thompson_ctd_loco whoi_glider_loco_we04 whoi_glider_loco_we11 ncom_icon_mbay sio_glider_asap_05_mean ausi_sauv03_loco sio_glider_asap_12_mean sio_glider_asap_11_mean sio_glider_asap_13_mean mbari_drifter_stella07 nrl_coamps_aosn_ocean uw_triaxus_aesop nps_aircraft_asap whoi_glider_asap_we02 ant_glider_01_ctd_ups ant_glider_02_ctd_ups whoi_glider_asap_we07 whoi_glider_asap_we09 whoi_glider_asap_we08 whoi_glider_asap_we12 roms_asap_mbay whoi_glider_asap_we10 whoi_glider_asap_we05 pt_sur_ctd_chavez hops_mbay_12h hops_anonuevo_12h hops_mbay hops_anonuevo sio_seasoar_aesop mbari_drifter_stella01 mbari_drifter_stella02 mbari_drifter_stella03 mbari_drifter_stella04 mbari_drifter_stella05 mbari_drifter_stella06 mbari_drifter_stella08 mbari_drifter_stella09 mbari_drifter_stella10 pt_sur_aesop nps_adcp1_asap nps_adcp2_asap calpoly_glider_ru06_asap uw_seaglider_106_ctd_ups uw_seaglider_022_ctd_ups uw_seaglider_023_ctd_ups new_horizon_svp pt_sur_udas

LOCO ASAP 2006/11/02 Bellingham & Godin

AESOP

UPS/PLUSNet

Other

Hits

Visits/Sites

Downloads

2006/08 Overall Usage

Teleconference 2006/11/02 Bellingham & Godin

Countries that accessed the site Argentina

Czech Republic

Italy

Australia

Denmark

Japan

Belgium

Estonia

Korea (South)

Sweden

Brazil

Finland

Mexico

Switzerland

Canada

France

Netherlands

Taiwan

Germany

Network

Hungary

New Zealand

Colombia

India

Norway

Costa Rica

Ireland

Poland

Chile Cocos Islands

Portugal Russian Federation

Turkey United Kingdom United States

Collaborative Portal Usage • Distributed observatory management achieved • Usage levels varied from day to day • Tools other than portal were sometimes used • More clarity in decisions on days when COOP tool was used heavily. 2006/11/02 Bellingham & Godin

Logins

Postings

Proposals

2006/08 Collaborative Portal Usage

Teleconference Missed Teleconference 2006/11/02 Bellingham & Godin

Press Day

No Call Scheduled

2006 Experiment Lessons • Standard visualizations are very useful • Virtual experiments are useful for deciding upon standard visualizations • Common format data needs formal, continuous QC • Need versioning on both source (raw) data and common format data • Collaborative tools should interoperate with familiar tools (e-mail lists, teleconference) 2006/11/02 Bellingham & Godin

Data System Thoughts • • • •

Add archive data (MUSE-2000, AOSN-2003) Add missing/revised MB2006 data Refine access controls Continue as “ad hoc observatory” – All available data to 500km west of Monterey

• Apply in other field programs and observatories • Allow real-time queries/visualizations (Metadata Oriented Query Assistant – MOQUA) 2006/11/02 Bellingham & Godin

User Feedback

• Data System – What worked / did not work for you? – What would you like to see in the future?

• Standard Products – can they be improved? • Portal – What worked / did not work for you? – What would you like to see in the future? 2006/11/02 Bellingham & Godin

COOP thoughts • Data system needs be brought into collaboration space – MOQUA as portlet? – Visualization portlet? – Produce movies of standard products? – Ability to share data by passing ‘pointers?’ – Ability to set up custom products or alerts?

COOP thoughts • Needs to scale gracefully as duration, data volume, and number of users grow – Provide overview of COOP activity (e.g., when is there lots of discussion, when is there a lot of interrogation of the site, what is the favorite page of the moment, etc. etc.) – Search capability (when did a particular individual post material to COOP) – Pivot off of something other than day – Allow users to break into separate ‘experiment spaces?’

COOP thoughts • Personalize COOP – News update for you when you log on (e.g. everyone voted on a new plan on the 12th, a particular person posted a message, etc.) – News flashes email to you (email version of above, or email version of update)? – Custom comparison products produced for you in your private workspace? – Alerts triggered off of observations or data (e.g. winds predicted to be from the NW tomorrow)

Entrain talent • Computer science community very excited about COOP • Visualization – e.g. Donna Cox, UW effort using World Wind • Work flow – e.g. KEPLER • Data systems – e.g. MBARI Shore Side Data System • Data systems – metadata standards…