Ken ichi Ohmichi NEC Solution Innovators, Ltd.

Transcription

1 Ken ichi Ohmichi NEC Solution Innovators, Ltd.

2 Agenda Who am I OpenStack OpenStack RESTful API Nova RESTful API Issues of Nova RESTful API Going Solutions Current Situation Summary 2

3 $ who am i 3

4 Software developer Join OpenStack community for its quality assurance since 2012 Working Areas OpenStack QA (Quality Assurance) OpenStack Compute (Nova) 4

5 5

6 OpenStack components OpenStack Neutron Network Nova Compute Glance Image Cinder Block storage Keystone Identity Horizon Dashboard 6

7 OpenStack community Rapid Growth A B C D E F G H I OpenStack Versions Companies Developers 7

8 Community development Importance of code review Can know the internal implementation Can know the trend Can get folks Importance of communication Daily : IRC or Weekly: IRC meeting Version: F2F at Design Summit 8

9 9

10 RESTful API Scalable and stateless Simple and consistent interfaces OpenStack provides many features through RESTful API OpenStack Server POST RESTful Client RESTful API Create a virtual machine VM 01 Hypervisor (kvm, etc) VM 02 10

11 Many RESTful client libraries are available for OpenStack Python -.NET Java - Dashboard and commands request operations via RESTful APIs Horizon Dashboard Commands $ nova.. $ cinder.. OpenStack Server RESTful Client library RESTful API

12 Each OpenStack component also is connected via RESTful APIs Neutron Network Nova Compute Nova RESTful API Glance Image Keystone Identity Horizon Dashboard Commands $ nova.. $ cinder..

13 13

14 OpenStack core functions Create a VM (virtual machine) Delete a VM Attach a volume to a VM Scalable Stable Current API version is v2 Nova v2 API contains 250+ APIs (Icehouse)

15 API usage Create a VM (virtual machine) URL Body RESTful Client { server : { name : VM01, flavorref : 42, imageref : d POST Nova v2 API Nova server VM01 Hypervisor (kvm, etc)

16 16

17 Inconsistent interfaces Inconsistent naming instance or server, tenant or project CamelCase, or snake_case { CamelCase instanceaction : [ { action : reboot, events : [], instance_uuid : b48316c5-[..], project_id : 147, [..] snake_case { server : { CamelCase accessipv4 :, accessipv6 :, id : 893c7791-[..], name : test-server, tenant_id : a54313c7-[..], [..] snake_case Show insance actions API Show server API 17

18 Inconsistent interfaces Return incorrect success codes Should return 202(ACCEPTED) but some APIs return 201(CREATED) even if not complete to create a resource yet. 18

19 Lacks of input validation Cause internal errors, output a lot of error logs in server, and return useless error messages Ex) Pass non-dict as metadata to create a VM Body RESTful Client { server : { name : VM01, flavorref : 42, imageref : d , metadata : non-dict POST The server has either erred or is incapable of performing the requested operation HTTP 500 Nova API server Nova v2 API Internal Error # this bug has been already fixed. 19

20 Lacks of input validation Ignore unexpected data which a client sends, and the request operates without some details which a client expects Body RESTful Client { server : { name : VM01, flavorref : 42, imageref : d , os:scheduler_fints : { group : group-01 POST typo HTTP 202 Nova API server Nova v2 API Create a VM without scheduler hints 20

21 21

22 Nova v3 API (Havana -> Juno) Re-define consistent naming Re-define consistent success codes Comprehensive input validation with JSONSchema V2 API Client Weak Validation Inconsistent interfaces Nova API Server v2.0 API (Current) Internal Error V3 API Client Strong Validation JSONSchema input validation v3 API (Experimen tal) Nova Internal implementation Consistent interfaces 22

23 { JSONSchema Common data format definition library Clear, human- and machine-readable data format definition { get_console_output : { length : 100 Correct request data Pass type : object, properties : { get_console_output : { type : object, properties : { length : { type : integer, minimum : -1, additionalproperties : False, required : [ get_console_output ] Data format definition with JSONSchema 23

24 { JSONSchema Contains basic validation features in nature Ex) minimum get_console_output : { length : -100 Invalid request data { Fail type : object, properties : { get_console_output : { type : object, properties : { length : { type : integer, minimum : -1, additionalproperties : False, required : [ get_console_output ] Data format definition with JSONSchema 24

25 { JSONSchema Can deny undefined parameters Ex) additionalproperties: False get_console_output : { length : 100, undefined : 100 Invalid request data { Fail type : object, properties : { get_console_output : { type : object, properties : { length : { type : integer, minimum : -1, additionalproperties : False, required : [ get_console_output ] Data format definition with JSONSchema 25

26 Use JSONSchema for input validation Strong validation, then avoid internal errors Return useful error message if a client sends invalid format data The server has either erred or is incapable of performing the requested operation Invalid input for field/attribute length. Value: foo. foo is not of type integer' before JSONSchema validation after JSONSchema validation 26

27 Use JSONSchema for input validation Clean up logic code by separating validation code Validation code also can be reduced try: min_count = int(str(min_count)) except ValueError: msg = _('min_count must be an integer value') raise exc.httpbadrequest(explanation=msg) if min_count < 1: msg = _('min_count must be > 0') raise exc.httpbadrequest(explanation=msg) Validation code of v2 API 'min_count': { 'type': 'integer', 'minimum': 1 Validation code of v3 API 27

28 Input validation development history The prototype proposal The first proposal in Portland summit The framework patch was approved, but merger failed Do not merge * 2

29 Input validation development history Investigated input validations of all API parameters and proposed necessary features The second proposal in Hong Kong summit. We have gotten a consensus The framework patch has been merged. Necessary to merge API definition patches still.

30 Input validation development status Merged patches: 14 Necessary to be reviewed/merged patches: 20

31 31

32 Stopped v3 API development Backward incompatibility against v2 API Large maintenance cost due to 250+ APIs for each API version Huge mail thread to keep/drop v3 API Nova API Server V2 API Client V3 API Client Backward incompatibility JSONSchema input validation v2.0 API (Current) Strong Validation v3 API (Experimen tal) Internal Error Nova Internal implementation 32

33 Proposed v2.1 API Translate v2 request/response and use v3 implementation Strong API validation Keep maintenance cost low Nova API Server V2 API Client Keep compatibility with strong validation V3 API Client Backward incompatibility v2.1 API (v2 v3 Translator) JSONSchema input validation v2.0 API (Current) Strong Validation v3 API (Experimen tal) Internal Error Nova Internal implementation 33

34 Consensuses after Atlanta Summit Implement v2.1 API in Juno cycle Implement JSONSchema validation for v2.1 API The future of v3 API is unclear now 34

35 35

36 Nova v2.1 API will be available after Juno Strong input validation The future of Nova v3 API is unclear Consistent interfaces Strong input validation Join OpenStack community Code review is important Communication is important 36

37 37