Roadmap. Introduction. Under the Hood. Solution Overview Energy Efficiency 101 Site Tour

Transcription

1 open-emv.com

2 The Team

3 Roadmap Introduction Solution Overview Energy Efficiency 101 Site Tour Under the Hood Basic Machine Configuration (AMIs) Software Configuration Database Structure The Web Application

4 The Solution User's Browser jquery OpenLayers Issue Tracking (FogBugz) VCS (beanstalk) User AWS EC2 m2.xlarge (high-memory) c1.medium (high-cpu) Database Applications Front-End Applications Zope Object Database ZEO Server Zope Client Zope Client Apache PostgreSQL Database PG Bouncer Zope Client Django (mod_wsgi) Development Machine mr.developer zc.buildout monitoring m1.small monitoring Monitoring Dashboard Developer

5 Why CA Energy Efficiency?

6 World Leader 1.5 Million Jobs $45 Billion in Payroll

7 How does that compare to the rest of the U.S?

8 California spends billions less on Electricity... has a ratio of GDP to Energy Consumed 68% higher... residents, per capita, pay lower utility bills

9 !"#$%&'()*+(,()-+()./0'-,1)./023)4()&5)&-)*(,()$6)(7.&('-)$6)8$2&5/,'&$9:)

10 Energy Efficiency and EMV

11 How Does CA do Energy Efficiency? CPUC established EE program cycle length and funding level CPUC provides $ to utilities to implement EE Programs Utilities report program accomplishments to CPUC

12 How Does CA do Energy Efficiency? CPUC hires independent EMV contractors to independently evaluate utility savings claims CPUC awards $ to the utilities based on savings accomplishments CPUC releases data publicly via website

13 Rinse and Repeat

14

15

16

19 AMI

20 AMI (Amazon Machine Image)

21 User's Browser jquery OpenLayers Issue Tracking (FogBugz) VCS (beanstalk) User AWS EC2 m2.xlarge (high-memory) Database Applications c1.medium (high-cpu) Front-End Applications Zope Object Database ZEO Server Zope Client Zope Client Apache PostgreSQL Database PG Bouncer Zope Client Django (mod_wsgi) Development Machine mr.developer zc.buildout monitoring m1.small monitoring Monitoring Dashboard Developer

22 m2.xlarge (high-memory) Database Applications c1.medium (high-cpu) Front-End Applications Zope Object Database ZEO Server Zope Client Zope Client Apache PostgreSQL Database PG Bouncer Zope Client Django (mod_wsgi)

23 Custom DB AMI m2.xlarge (high-memory) Database Applications Zope Object Database ZEO Server PostgreSQL Database PG Bouncer monitoring

24 Custom DB AMI m2.xlarge (high-memory) Database Applications PostgreSQL Database PG Bouncer monitoring

25 Custom DB AMI m2.xlarge (high-memory) Ubuntu (Maverick) Database Applications PostgreSQL Database PG Bouncer monitoring

26 Custom DB AMI m2.xlarge (high-memory) Database Applications Ubuntu (Maverick) 64-bit PostgreSQL Database PG Bouncer monitoring

27 Custom DB AMI m2.xlarge (high-memory) Database Applications Ubuntu (Maverick) 64-bit Kernel Settings PostgreSQL Database PG Bouncer monitoring

28 Custom DB AMI m2.xlarge (high-memory) Ubuntu (Maverick) Database Applications PostgreSQL Database monitoring PG Bouncer 64-bit Kernel Settings kernel.shmmax = 8GB kernel.shmall = 8GB/4096 /etc/sysctl.conf

29 Custom DB AMI m2.xlarge (high-memory) Database Applications Ubuntu (Maverick) 64-bit Kernel Settings PostgreSQL Database PG Bouncer Spatial Database monitoring

30 Custom DB AMI m2.xlarge (high-memory) Database Applications Ubuntu (Maverick) 64-bit Kernel Settings PostgreSQL Database monitoring PG Bouncer Spatial Database apt-get install... PostgreSQL Geos GDAL manual install postgis 1.5.2

31 Custom DB AMI m2.xlarge (high-memory) Database Applications Ubuntu (Maverick) 64-bit Kernel Settings PostgreSQL Database monitoring PG Bouncer Spatial Database Connection Pooling

32 Custom DB AMI m2.xlarge (high-memory) Database Applications Ubuntu (Maverick) 64-bit Kernel Settings PostgreSQL Database PG Bouncer Spatial Database monitoring Connection Pooling apt-get install... pgbouncer 1.3.3

33 Custom DB AMI m2.xlarge (high-memory) Database Applications Ubuntu (Maverick) 64-bit Kernel Settings PostgreSQL Database monitoring PG Bouncer Spatial Database Connection Pooling Monitoring

34 Custom DB AMI m2.xlarge (high-memory) Database Applications Ubuntu (Maverick) 64-bit Kernel Settings PostgreSQL Database monitoring PG Bouncer Spatial Database Connection Pooling Monitoring apt-get install... monit munin-node

35 Custom Web App AMI c1.medium (high-cpu) Front-End Applications Zope Client Apache Zope Client Zope Client Django (mod_wsgi) monitoring

36 Custom Web App AMI c1.medium (high-cpu) Front-End Applications Zope Client Apache Zope Client Zope Client Django (mod_wsgi) monitoring

37 Custom Web App AMI c1.medium Ubuntu (Maverick) c1.medium (high-cpu) Front-End Applications Zope Client Apache Zope Client Zope Client Django (mod_wsgi) monitoring

38 Custom Web App AMI Ubuntu (Maverick) 32-bit c1.medium (high-cpu) Front-End Applications Zope Client Zope Client Apache Zope Client Django (mod_wsgi) monitoring

39 Custom Web App AMI Ubuntu (Maverick) 32-bit Apache2 c1.medium (high-cpu) Front-End Applications Zope Client Zope Client Zope Client Apache Django (mod_wsgi) monitoring

40 Custom Web App AMI Ubuntu (Maverick) 32-bit Apache2 mod_wsgi c1.medium (high-cpu) Front-End Applications Zope Client Zope Client Zope Client monitoring Apache Django (mod_wsgi)

41 Custom Web App AMI Ubuntu (Maverick) 32-bit Apache2 mod_wsgi Monitoring c1.medium (high-cpu) Front-End Applications Zope Client Zope Client Zope Client monitoring Apache Django (mod_wsgi)

42 We ll make these public

43 We ll make these public once I ve polished them up a bit :)

44 Additional Software

45 Additional Software Back-End

46 Additional Software Back-End ZODB

47 Additional Software Back-End ZODB ZEO Server

48 Additional Software Back-End ZODB ZEO Server Front-End

49 Additional Software Back-End ZODB ZEO Server Front-End Django

50 Additional Software Back-End ZODB ZEO Server Front-End Django Plone

51 Additional Software Back-End ZODB ZEO Server Front-End Django Plone Add-Ons

52 Additional Software Back-End ZODB ZEO Server Front-End Django Plone Add-Ons Zope Clients

53 And Configuration Back-End ZODB ZEO Server Front-End Django Plone Add-Ons Zope Clients

54 And Configuration Back-End ZODB ZEO Server Front-End Django Plone Add-Ons Zope Clients

55 And Configuration Back-End ZODB ZEO Server PostgreSQL Front-End Django Plone Add-Ons Zope Clients

56 And Configuration Back-End ZODB ZEO Server PostgreSQL Front-End Django Plone Add-Ons Zope Clients

57 And Configuration Back-End ZODB ZEO Server PostgreSQL Front-End Django Plone Add-Ons Zope Clients Apache

58 And Configuration Back-End ZODB ZEO Server PostgreSQL Front-End Django Plone Add-Ons Zope Clients Apache

59 And Configuration Back-End ZODB ZEO Server PostgreSQL Front-End Django Plone Add-Ons Zope Clients Apache Monit

60 And Configuration Back-End ZODB ZEO Server PostgreSQL Apache Front-End Django Plone Add-Ons Zope Clients Replicate On Development Machine(s) Monit

61 Photo by EvelynGiggles via Flickr using a CC-BY license.

62 This is a job for Superman!

63 This is a job for Buildout!

64 Wait what?

65 zc.buildout

66 developed for

67 used extensively by

68 also gaining traction with

69 not limited to python

70 dependencies

71 dependencies $ apt-get install python-setuptools

72 dependencies $ apt-get install python-setuptools $ easy-install distribute

73 dependencies $ apt-get install python-setuptools $ easy-install distribute $ apt-get install python-dev

74 The Basics

75 The Basics $ ls basic-buildout bootstrap.py buildout.cfg

76 The Basics

77 The Basics $ more buildout.cfg [buildout] parts = foo [foo] recipe = collective.recipe.cmd cmd = echo foo

78 Build It

79 Run It

80 Photo by Rich Moore via Flickr using a CC-BY license.

81 let s scale this up

82 lots of cfg, and more

83 base.cfg

84 pg_parts.cfg

85 A Closer Look

86 Tie it all together...

87 dev.cfg

88 or, for deployment...

89 backend.cfg

90 Buildout Advantages

91 Buildout Advantages Easy to write

92 Buildout Advantages Easy to write Lots of recipes, examples

93 Buildout Advantages Easy to write Lots of recipes, examples Manage complexity

94 Buildout Advantages Easy to write Lots of recipes, examples Manage complexity Duplicate across platforms

95 Buildout Advantages Easy to write Lots of recipes, examples Manage complexity Duplicate across platforms Deploy rapidly

96 Buildout Advantages Easy to write Lots of recipes, examples Manage complexity Duplicate across platforms Deploy rapidly Versioning of software setup and config

97 not without issues

98 not without issues Can be network-dependent

99 not without issues Can be network-dependent Interconnected parts mean complexity

100 not without issues Can be network-dependent Interconnected parts mean complexity extends can be tricky to manage

101 use it in joy

102 Data ingest

103 Data dimensions

104 The Web Application

105 8 Axes of Data

106 8 Axes of Data Time Frame (lifecycle or first-year)

107 8 Axes of Data Time Frame (lifecycle or first-year) Savings Type (but cost too)

108 8 Axes of Data Time Frame (lifecycle or first-year) Savings Type (but cost too) Year

109 8 Axes of Data Time Frame (lifecycle or first-year) Savings Type (but cost too) Year Utility

110 8 Axes of Data Time Frame (lifecycle or first-year) Savings Type (but cost too) Year Utility Program Group

111 8 Axes of Data Time Frame (lifecycle or first-year) Savings Type (but cost too) Year Utility Program Group Measure Group

112 8 Axes of Data Time Frame (lifecycle or first-year) Savings Type (but cost too) Year Utility Program Group Measure Group Building Type

113 8 Axes of Data Time Frame (lifecycle or first-year) Savings Type (but cost too) Year Utility Program Group Measure Group Building Type Geography

114 only 3 are required

115 8 Axes of Data Time Frame (lifecycle or first-year) Savings Type (but cost too) Year Utility Program Group Measure Group Building Type Geography

116 Show me the [timeframe] [savings type] by [geography]

117 Choices drawn from the distinct values for a specific table column Show me the [timeframe] [savings type] by [geography]

118 Choices drawn from the distinct values for a specific table column Show me the [timeframe] [savings type] by [geography] Choices drawn from a set of table columns (not the contained values)

119 Choices drawn from the distinct values for a specific table column Choices made up but reference table-borne data indirectly Show me the [timeframe] [savings type] by [geography] Choices drawn from a set of table columns (not the contained values)

120 Show me the [timeframe] [savings type] by [geography]

121 Show me the [timeframe] [savings type] by [geography] for [year] for [utility] for [program group] for [measure group] for [building type]

122 Show me the [timeframe] [savings type] by [geography] for [building type] for [year] for [utility] for [program group] for [measure group]

123 Show me the [timeframe] [savings type] by [geography] for [building type] for [year] for [utility] for [program group] for [measure group]

124 the sentence interface

125 the sentence interface Thanks to FogBugz for the inspiration

126 a Django package

127 a Django package A Form

128 a Django package A Form A Field

129 a Django package A Form A Field A Widget

130 We ll make this public

131 We ll make this public once I ve polished it up a bit :)