Mailing List Archive

Hello Luis,

I presented a blueprint last week [1] which proposes to clearly
differentiate metering from the overall billing process. It is my
understanding that billing is too complex a beast to be solved for each
requirement in a satisfactory way and have therefore proposed that we
should first concentrate on collecting the informations necessary for
billing systems to pull their information from.

The blueprint, and the discussion we had at the summit last week,
outlines a proposed architecture but has, so far, left open the
component choices to implement it. It is our proposal to start a a
weekly meeting from that week to start selecting the components to
deliver this. You are more than welcome to join the project if you want.

[1] http://wiki.openstack.org/EfficientMetering

Best regards,
Nick

On 04/22/2012 08:50 PM, Luis Gervaso wrote:
> Hi,
>
> I want to share the architecture i am developing in order to perform the
> monitorig / billing OpenStack support:
>
> 1. AMQP Client which listen to RabbitMQ / QPid (this should be
> interchangeable) (Own Stuff or ServiceMix / Camel)
>
> 2. Events should be stored on a NoSQL document oriented database (I
> think mongodb is perfect, since we can query in a super easy fashion)
>
> 3a. The monitoring system can pull/push MongoDB
>
> 3b. The billing system can pull to create invoices
>
> 4. A mediation EIP should be necessary to integrate a billing/monitoring
> product. (ServiceMix / Camel)
>
> This is to receive your feedback. So please, critics are welcome!
>
> Cheers!
>
> --
> -------------------------------------------
> Luis Alberto Gervaso Martin
> Woorea Solutions, S.L
> CEO & CTO
> mobile: (+34) 627983344
> luis@ <mailto:luis.gervaso@gmail.com>woorea.es <http://woorea.es/>
>
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp

Hi,

Although metering and billing always comes together for the deployment, for the sake of clarity, I also think metering should be a separate project from the billing, especially in openstack.
(As you mentioned it, billing is complex and has too many different requirements per provider)

Anyway, I am really interested in weekly meeting you mentioned in the email.
How can I join the meeting? Is it irc-meeting?, mailing-list?, or else?


--
Jaesuk Ahn, Ph.D.
Team Leader | Cloud OS Dev. Team
Cloud Infrastructure Department
KT (Korea Telecom)
T. +82-10-9888-0328 | F. +82-303-0993-5340
Active member on OpenStack Korea Community


Apr 23, 2012, 4:09 PM, Nick Barcet ÀÛ¼º:

> Hello Luis,
>
> I presented a blueprint last week [1] which proposes to clearly
> differentiate metering from the overall billing process. It is my
> understanding that billing is too complex a beast to be solved for each
> requirement in a satisfactory way and have therefore proposed that we
> should first concentrate on collecting the informations necessary for
> billing systems to pull their information from.
>
> The blueprint, and the discussion we had at the summit last week,
> outlines a proposed architecture but has, so far, left open the
> component choices to implement it. It is our proposal to start a a
> weekly meeting from that week to start selecting the components to
> deliver this. You are more than welcome to join the project if you want.
>
> [1] http://wiki.openstack.org/EfficientMetering
>
> Best regards,
> Nick
>
> On 04/22/2012 08:50 PM, Luis Gervaso wrote:
>> Hi,
>>
>> I want to share the architecture i am developing in order to perform the
>> monitorig / billing OpenStack support:
>>
>> 1. AMQP Client which listen to RabbitMQ / QPid (this should be
>> interchangeable) (Own Stuff or ServiceMix / Camel)
>>
>> 2. Events should be stored on a NoSQL document oriented database (I
>> think mongodb is perfect, since we can query in a super easy fashion)
>>
>> 3a. The monitoring system can pull/push MongoDB
>>
>> 3b. The billing system can pull to create invoices
>>
>> 4. A mediation EIP should be necessary to integrate a billing/monitoring
>> product. (ServiceMix / Camel)
>>
>> This is to receive your feedback. So please, critics are welcome!
>>
>> Cheers!
>>
>> --
>> -------------------------------------------
>> Luis Alberto Gervaso Martin
>> Woorea Solutions, S.L
>> CEO & CTO
>> mobile: (+34) 627983344
>> luis@ <mailto:luis.gervaso@gmail.com>woorea.es <http://woorea.es/>
>>
>>
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help : https://help.launchpad.net/ListHelp
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp

On 04/23/2012 11:17 AM, Ahn, Jaesuk wrote:
> Hi,
>
> Although metering and billing always comes together for the deployment,
> for the sake of clarity, I also think metering should be a separate
> project from the billing, especially in openstack.
> (As you mentioned it, billing is complex and has too many different
> requirements per provider)
>
> Anyway, I am really interested in weekly meeting you mentioned in the
> email.
> How can I join the meeting? Is it irc-meeting?, mailing-list?, or else?

Of course you can join. It will be a weekly irc meeting and I am about
to send a doodle with multiple time choices to everyone that mentioned
being interested in joining this project at the summit, and have
included your email now. The first meeting will have for objective to
finalize a project name and validate a road-map of decisions that need
to be made.

Thanks a lot,

Nick

> --
> *Jaesuk Ahn*, Ph.D.
> Team Leader | Cloud OS Dev. Team
> Cloud Infrastructure Department
> KT (Korea Telecom)
> *T. +82-10-9888-0328 | F. +82-303-0993-5340*
> *Active member on **OpenStack Korea Community* <http://www.openstack.or.kr/>
>
>
> Apr 23, 2012, 4:09 PM, Nick Barcet ÀÛ¼º:
>
>> Hello Luis,
>>
>> I presented a blueprint last week [1] which proposes to clearly
>> differentiate metering from the overall billing process. It is my
>> understanding that billing is too complex a beast to be solved for each
>> requirement in a satisfactory way and have therefore proposed that we
>> should first concentrate on collecting the informations necessary for
>> billing systems to pull their information from.
>>
>> The blueprint, and the discussion we had at the summit last week,
>> outlines a proposed architecture but has, so far, left open the
>> component choices to implement it. It is our proposal to start a a
>> weekly meeting from that week to start selecting the components to
>> deliver this. You are more than welcome to join the project if you want.
>>
>> [1] http://wiki.openstack.org/EfficientMetering
>>
>> Best regards,
>> Nick
>>
>> On 04/22/2012 08:50 PM, Luis Gervaso wrote:
>>> Hi,
>>>
>>> I want to share the architecture i am developing in order to perform the
>>> monitorig / billing OpenStack support:
>>>
>>> 1. AMQP Client which listen to RabbitMQ / QPid (this should be
>>> interchangeable) (Own Stuff or ServiceMix / Camel)
>>>
>>> 2. Events should be stored on a NoSQL document oriented database (I
>>> think mongodb is perfect, since we can query in a super easy fashion)
>>>
>>> 3a. The monitoring system can pull/push MongoDB
>>>
>>> 3b. The billing system can pull to create invoices
>>>
>>> 4. A mediation EIP should be necessary to integrate a billing/monitoring
>>> product. (ServiceMix / Camel)
>>>
>>> This is to receive your feedback. So please, critics are welcome!
>>>
>>> Cheers!
>>>
>>> --
>>> -------------------------------------------
>>> Luis Alberto Gervaso Martin
>>> Woorea Solutions, S.L
>>> CEO & CTO
>>> mobile: (+34) 627983344
>>> luis@ <mailto:luis.gervaso@gmail.com>woorea.es <http://woorea.es/>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to : openstack@lists.launchpad.net
>>> <mailto:openstack@lists.launchpad.net>
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help : https://help.launchpad.net/ListHelp
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> <mailto:openstack@lists.launchpad.net>
>> Unsubscribe : https://launchpad.net/~openstack
>> More help : https://help.launchpad.net/ListHelp
>
>
>

Hi!

We have developed Nova Billing v2. Its documentation is currently available
at http://aababilov.github.com/nova-billing-doc.github.com/. The
documentation includes a glossary and architecture and API descriptions.

Nova Billing v2 is a totally new solution. Its API and architecture were
rewritten from scratch. The new Nova Billing introduces extensible
modularized system with separate components.

Nova Billing v2 can charge arbitrary resources according to custom billing
schemas and, naturally, with different tariffs.

Nova Billing v2 does not depend on any UI for OpenStack Cloud (neither
OpenStack Dashboard nor any other solution). It does not require a
particular OpenStack release. Provided components allow integration with
diablo and essex and this list can be extended.

Nova Billing v2 is event-driven and does not consumes system resources for
periodical polling.

Nova Billing v2 uses a RESTful protocol, so it is easy to integrate it with
miscellaneous user clients and to add third-party components notifying
about arbitrary events.
Alessio Ababilov,
Software Engineer
Grid Dynamics

On 04/23/2012 04:39 PM, Alexey Ababilov wrote:
> Hi!
>
> We have developed Nova Billing v2. Its documentation is currently
> available at http://aababilov.github.com/nova-billing-doc.github.com/.
> The documentation includes a glossary and architecture and API descriptions.
>
> Nova Billing v2 is a totally new solution. Its API and architecture were
> rewritten from scratch. The new Nova Billing introduces extensible
> modularized system with separate components.
>
> Nova Billing v2 can charge arbitrary resources according to custom
> billing schemas and, naturally, with different tariffs.
>
> Nova Billing v2 does not depend on any UI for OpenStack Cloud (neither
> OpenStack Dashboard nor any other solution). It does not require a
> particular OpenStack release. Provided components allow integration with
> diablo and essex and this list can be extended.
>
> Nova Billing v2 is event-driven and does not consumes system resources
> for periodical polling.
>
> Nova Billing v2 uses a RESTful protocol, so it is easy to integrate it
> with miscellaneous user clients and to add third-party components
> notifying about arbitrary events.
>
> Alessio Ababilov,
> Software Engineer
> Grid Dynamics

Excellent! It sounds from the architecture that your work will easily be
integrated into a the generic OpenStack metering solution that we are
currently defining. The main goal of our design is to dissociate the
metering from the billing part and to offer extensibility to all current
and future components of OpenStack. Would you care to help us define
this architecture?

Thanks,
Nick

On Sun, 2012-04-22 at 20:50 +0200, Luis Gervaso wrote:
> I want to share the architecture i am developing in order to perform
> the monitorig / billing OpenStack support:
>
>
> 1. AMQP Client which listen to RabbitMQ / QPid (this should be
> interchangeable) (Own Stuff or ServiceMix / Camel)
>
>
> 2. Events should be stored on a NoSQL document oriented database (I
> think mongodb is perfect, since we can query in a super easy fashion)

Except for the use of MongoDB, the above seems to me to be almost
identical to the notifications system already in Nova, which Yagi
consumes. Have you looked at our existing notifications? Yagi? One or
both might solve at least parts of your problem…

--
Kevin L. Mitchell <kevin.mitchell@rackspace.com>


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

This already exists in trunk. The Notification system was designed specifically to feed billing and monitoring systems.

Basically, we don't want Nova/Glance/etc to be in the business of trying to determine billing logic, since it is different for pretty much everyone, so we just emit notifications to a queue and the interested pull what they want, and aggregate according to their own rules.

On Apr 22, 2012, at 1:50 PM, Luis Gervaso wrote:

Hi,

I want to share the architecture i am developing in order to perform the monitorig / billing OpenStack support:

1. AMQP Client which listen to RabbitMQ / QPid (this should be interchangeable) (Own Stuff or ServiceMix / Camel)

2. Events should be stored on a NoSQL document oriented database (I think mongodb is perfect, since we can query in a super easy fashion)

We have an existing system called Yagi (https://github.com/Cerberus98/yagi/) that listens to the notification queues and persists events to a Redis database. It then provides feeds as ATOM formatted documents that a billing system can pull to aggregate data, It also can support PubSub notification of clients thru the pubsubhubub protocol, and push events to a long-term archiving store thru the AtomPub protocol.

That said, the notification system outputs its events as JSON, so it should be easy to pipe into a json document-oriented db if that's what you need. (we only use ATOM because we have a atom-based archiving/search/aggregation engine (it's open source: http://atomhopper.org/ ) our in-house systems already plug into. )




3a. The monitoring system can pull/push MongoDB

3b. The billing system can pull to create invoices

4. A mediation EIP should be necessary to integrate a billing/monitoring product. (ServiceMix / Camel)

This is to receive your feedback. So please, critics are welcome!

Cheers!

--
-------------------------------------------
Luis Alberto Gervaso Martin
Woorea Solutions, S.L
CEO & CTO
mobile: (+34) 627983344
luis@<mailto:luis.gervaso@gmail.com>woorea.es<http://woorea.es/>


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net<mailto:openstack@lists.launchpad.net>
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

--
Monsyne M. Dragon
OpenStack/Nova
cell 210-441-0965
work x 5014190

Nick, I want contribute in the discussion group.

I have seen yagi implementation, and i like it. I like the protocol.

But I think that it's a protocol that fits very well when no sensible data
is involved (content feeds). When money, resources, companies are involved
i think we need to use a more elaborated schema.

I don't like the 3rd parties have to pull, my proporsal here is that 3rd
parties can be entities externalized outside of the datacenter.

1. i don't want to make the accounting public accessible for security
reasons.
2. 3rd parties can't be coupled to openstack, i want to feed them. I think
this is the key.

What i realized is that the listener needs to feed the billing system
following the rules/api of the selected billing system.

Scenario:

1. A company has an account on a billing provider
2. A company should define the pricing model in the billing system, outside
openstack (i.e: inside zuora, jbilling, recurly, dough)
3. When a message arrives to the listener the listener has to perform the
following actions:

3a: transform and store the message in a accounting system (restful api)
3b (optional): transform and store the message in a billing system (restful
api) example: [
http://knowledgecenter.zuora.com/C_Zuora_User_Guides/A_Billing_and_Payments/A_Z-Billing
]
3c (optional): perform monitoring (currently evaluating sigar functionality)

3a: is openstack related

3b, 3c, ...: are hooks

4a. A company can query the billing system to gather the bills (no
openstack dependencies)
4b. A company can query the monitoring system (no openstack dependencies)

Regarding MongoDB, I recommend a document oriented database where we can
query with criteria. Redis is key-value oriented.

Cheers!

On Mon, Apr 23, 2012 at 4:51 PM, Kevin L. Mitchell <
kevin.mitchell@rackspace.com> wrote:

> On Sun, 2012-04-22 at 20:50 +0200, Luis Gervaso wrote:
> > I want to share the architecture i am developing in order to perform
> > the monitorig / billing OpenStack support:
> >
> >
> > 1. AMQP Client which listen to RabbitMQ / QPid (this should be
> > interchangeable) (Own Stuff or ServiceMix / Camel)
> >
> >
> > 2. Events should be stored on a NoSQL document oriented database (I
> > think mongodb is perfect, since we can query in a super easy fashion)
>
> Except for the use of MongoDB, the above seems to me to be almost
> identical to the notifications system already in Nova, which Yagi
> consumes. Have you looked at our existing notifications? Yagi? One or
> both might solve at least parts of your problem…
>
> --
> Kevin L. Mitchell <kevin.mitchell@rackspace.com>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
>



--
-------------------------------------------
Luis Alberto Gervaso Martin
Woorea Solutions, S.L
CEO & CTO
mobile: (+34) 627983344
luis@ <luis.gervaso@gmail.com>woorea.es

On Sun, Apr 22, 2012 at 5:29 PM, Brian Schott <
brian.schott@nimbisservices.com> wrote:

> Dough is a proposed billing service. There was a session at Folsom design
> summit. This is a practical project for an OpenStack provider with test
> code on github.
> http://summit.openstack.org/sessions/view/37
>
> "
> The billing system consists of three components.
> 1) API Server, which receives subscribe/unsubscribe and usage query
> requests.
> 2) Farmer, which periodically checks all subscriptions and requests
> billing to the collector.
> 3) Collector, gets work from the farmer and uses python-novaclient,
> kanyun-client, swift-client to retrieve usage information of a product the
> tenant has subscribed and charges money to the accordingly.
> "
> Kanyun is an OpenStack monitoring tool, but I don't see documentation in
> the code tree (but I've just pulled it to see).
> https://github.com/lzyeval
>
> Dough had mixed reviews during the session, but that I think was because
> it came across as a billing system triggered solely by Horizon requests and
> was a polled architecture.
>

I also had some reservations about Dough because it seemed brittle. For
example, it was my impression that if an instance creation operation failed
for some reason, the operator had to manually correct the billing data.


> Given what we have today, I'd implement my own billing/metering system
> exactly the same way. In fact, I have. Our own e-commerce system at
> Nimbis works this way with EC2 and OpenStack presently because the only
> option available is polling periodically and logging the usage data.
>

The fact that it only worked through Horizon was the biggest issue for us.
That means if we expose the API to our users, instances created via the API
but not the Horizon dashboard would not be billed. I understood the
decision to be a proof-of-concept, but an event-based model for collecting
the data seems much more robust and flexible.


> Where I'd like to see OpenStack go is metering service that is fully
> asynchronous and event driven so that I only need to hit the API service
> when I'm generating an invoice, rather than once per minute because I don't
> know when an instance was terminated by the user or just crashed.
>
> -------------------------------------------------
> Brian Schott, CTO
> Nimbis Services, Inc.
> brian.schott@nimbisservices.com
> ph: 443-274-6064 fx: 443-274-6060
>
>
>
> On Apr 22, 2012, at 4:57 PM, Endre Karlson wrote:
>
> What is Dough then compared to what you want to do ?
>
> 2012/4/22 Endre Karlson <endre.karlson@gmail.com>
>
>> What is Dough then ?
>>
>>
>> 2012/4/22 Brian Schott <brian.schott@nimbisservices.com>
>>
>>> I see this blueprint for metering, but none for Dough currently.
>>> http://wiki.openstack.org/EfficientMetering
>>>
>>> Here are the Dough slides, however:
>>> http://www.slideshare.net/lzyeval/dough-openstack-billing-project
>>>
>>> We collectively need to talk more about the user scenarios, because I
>>> don't think you can just put a decorator around the API rpc calls and get
>>> an accurate picture of what to bill for later. There are metered things
>>> like bandwidth or IOPS, events that happen outside of the API (shutdown
>>> -h), and it is hard to predict what a reseller will want to charge for. It
>>> is better to put a metering system in that can handle many billing
>>> configurations.
>>>
>>>
>>> -------------------------------------------------
>>> Brian Schott, CTO
>>> Nimbis Services, Inc.
>>> brian.schott@nimbisservices.com
>>> ph: 443-274-6064 fx: 443-274-6060
>>>
>>>
>>>
>>> On Apr 22, 2012, at 3:36 PM, Luis Gervaso wrote:
>>>
>>> Dough is the proposed billing platform/product (where the billing rules
>>> live), isn't it?
>>>
>>> I don't know Dough enough, so please me correct me if i'm wrong.
>>>
>>> I'm trying to define a generic/agnostic integration process, obviously
>>> where Dough
>>> can fit perfectly. I would like it become part to the reference
>>> architecture.
>>>
>>> Option 1) [3b in the arch proposed]
>>>
>>> Dough should pull NoSQL
>>>
>>> Option 2)
>>>
>>> A Mediator can feed Dough
>>>
>>>
>>> On Sun, Apr 22, 2012 at 9:13 PM, Endre Karlson <endre.karlson@gmail.com>wrote:
>>>
>>>> What about using the Dough project?
>>>>
>>>> Endre.
>>>>
>>>>
>>>> 2012/4/22 Endre Karlson <endre.karlson@gmail.com>
>>>>
>>>>> What about using the Dough project ?
>>>>>
>>>>> Endre.
>>>>>
>>>>> 2012/4/22 Luis Gervaso <luis@woorea.es>
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I want to share the architecture i am developing in order to perform
>>>>>> the monitorig / billing OpenStack support:
>>>>>>
>>>>>> 1. AMQP Client which listen to RabbitMQ / QPid (this should be
>>>>>> interchangeable) (Own Stuff or ServiceMix / Camel)
>>>>>>
>>>>>> 2. Events should be stored on a NoSQL document oriented database (I
>>>>>> think mongodb is perfect, since we can query in a super easy fashion)
>>>>>>
>>>>>> 3a. The monitoring system can pull/push MongoDB
>>>>>>
>>>>>> 3b. The billing system can pull to create invoices
>>>>>>
>>>>>> 4. A mediation EIP should be necessary to integrate a
>>>>>> billing/monitoring product. (ServiceMix / Camel)
>>>>>>
>>>>>> This is to receive your feedback. So please, critics are welcome!
>>>>>>
>>>>>> Cheers!
>>>>>>
>>>>>> --
>>>>>> -------------------------------------------
>>>>>> Luis Alberto Gervaso Martin
>>>>>> Woorea Solutions, S.L
>>>>>> CEO & CTO
>>>>>> mobile: (+34) 627983344
>>>>>> luis@ <luis.gervaso@gmail.com>woorea.es
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Mailing list: https://launchpad.net/~openstack
>>>>>> Post to : openstack@lists.launchpad.net
>>>>>> Unsubscribe : https://launchpad.net/~openstack
>>>>>> More help : https://help.launchpad.net/ListHelp
>>>>>>
>>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~openstack
>>>> Post to : openstack@lists.launchpad.net
>>>> Unsubscribe : https://launchpad.net/~openstack
>>>> More help : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>>
>>> --
>>> -------------------------------------------
>>> Luis Alberto Gervaso Martin
>>> Woorea Solutions, S.L
>>> CEO & CTO
>>> mobile: (+34) 627983344
>>> luis@ <luis.gervaso@gmail.com>woorea.es
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to : openstack@lists.launchpad.net
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help : https://help.launchpad.net/ListHelp
>>>
>>>
>>>
>>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
>
>

Is there a document somewhere on what events the services emit?

-------------------------------------------------
Brian Schott, CTO
Nimbis Services, Inc.
brian.schott@nimbisservices.com
ph: 443-274-6064 fx: 443-274-6060



On Apr 23, 2012, at 12:39 PM, Monsyne Dragon wrote:

> This already exists in trunk. The Notification system was designed specifically to feed billing and monitoring systems.
>
> Basically, we don't want Nova/Glance/etc to be in the business of trying to determine billing logic, since it is different for pretty much everyone, so we just emit notifications to a queue and the interested pull what they want, and aggregate according to their own rules.
>
> On Apr 22, 2012, at 1:50 PM, Luis Gervaso wrote:
>
>> Hi,
>>
>> I want to share the architecture i am developing in order to perform the monitorig / billing OpenStack support:
>>
>> 1. AMQP Client which listen to RabbitMQ / QPid (this should be interchangeable) (Own Stuff or ServiceMix / Camel)
>
>> 2. Events should be stored on a NoSQL document oriented database (I think mongodb is perfect, since we can query in a super easy fashion)
>
> We have an existing system called Yagi (https://github.com/Cerberus98/yagi/) that listens to the notification queues and persists events to a Redis database. It then provides feeds as ATOM formatted documents that a billing system can pull to aggregate data, It also can support PubSub notification of clients thru the pubsubhubub protocol, and push events to a long-term archiving store thru the AtomPub protocol.
>
> That said, the notification system outputs its events as JSON, so it should be easy to pipe into a json document-oriented db if that's what you need. (we only use ATOM because we have a atom-based archiving/search/aggregation engine (it's open source: http://atomhopper.org/ ) our in-house systems already plug into. )
>
>
>
>>
>> 3a. The monitoring system can pull/push MongoDB
>>
>> 3b. The billing system can pull to create invoices
>>
>> 4. A mediation EIP should be necessary to integrate a billing/monitoring product. (ServiceMix / Camel)
>>
>> This is to receive your feedback. So please, critics are welcome!
>>
>> Cheers!
>>
>> --
>> -------------------------------------------
>> Luis Alberto Gervaso Martin
>> Woorea Solutions, S.L
>> CEO & CTO
>> mobile: (+34) 627983344
>> luis@woorea.es
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help : https://help.launchpad.net/ListHelp
>
> --
> Monsyne M. Dragon
> OpenStack/Nova
> cell 210-441-0965
> work x 5014190
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp

I have been looking at : http://wiki.openstack.org/SystemUsageData

On Mon, Apr 23, 2012 at 7:35 PM, Brian Schott <
brian.schott@nimbisservices.com> wrote:

> Is there a document somewhere on what events the services emit?
>
> -------------------------------------------------
> Brian Schott, CTO
> Nimbis Services, Inc.
> brian.schott@nimbisservices.com
> ph: 443-274-6064 fx: 443-274-6060
>
>
>
> On Apr 23, 2012, at 12:39 PM, Monsyne Dragon wrote:
>
> This already exists in trunk. The Notification system was designed
> specifically to feed billing and monitoring systems.
>
> Basically, we don't want Nova/Glance/etc to be in the business of trying
> to determine billing logic, since it is different for pretty much everyone,
> so we just emit notifications to a queue and the interested pull what they
> want, and aggregate according to their own rules.
>
> On Apr 22, 2012, at 1:50 PM, Luis Gervaso wrote:
>
> Hi,
>
> I want to share the architecture i am developing in order to perform the
> monitorig / billing OpenStack support:
>
> 1. AMQP Client which listen to RabbitMQ / QPid (this should be
> interchangeable) (Own Stuff or ServiceMix / Camel)
>
>
>
> 2. Events should be stored on a NoSQL document oriented database (I think
> mongodb is perfect, since we can query in a super easy fashion)
>
>
> We have an existing system called Yagi (
> https://github.com/Cerberus98/yagi/) that listens to the notification
> queues and persists events to a Redis database. It then provides feeds as
> ATOM formatted documents that a billing system can pull to aggregate data,
> It also can support PubSub notification of clients thru the pubsubhubub
> protocol, and push events to a long-term archiving store thru the AtomPub
> protocol.
>
> That said, the notification system outputs its events as JSON, so it
> should be easy to pipe into a json document-oriented db if that's what you
> need. (we only use ATOM because we have a atom-based
> archiving/search/aggregation engine (it's open source:
> http://atomhopper.org/ ) our in-house systems already plug into. )
>
>
>
>
> 3a. The monitoring system can pull/push MongoDB
>
> 3b. The billing system can pull to create invoices
>
> 4. A mediation EIP should be necessary to integrate a billing/monitoring
> product. (ServiceMix / Camel)
>
> This is to receive your feedback. So please, critics are welcome!
>
> Cheers!
>
> --
> -------------------------------------------
> Luis Alberto Gervaso Martin
> Woorea Solutions, S.L
> CEO & CTO
> mobile: (+34) 627983344
> luis@ <luis.gervaso@gmail.com>woorea.es
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
>
>
> --
> Monsyne M. Dragon
> OpenStack/Nova
> cell 210-441-0965
> work x 5014190
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
>
>
>


--
-------------------------------------------
Luis Alberto Gervaso Martin
Woorea Solutions, S.L
CEO & CTO
mobile: (+34) 627983344
luis@ <luis.gervaso@gmail.com>woorea.es

If you wanted to use Yagi, it would be trivial to add a JSON only notifier to Yagi. If you're interested and need a hand, feel free to hit Dragon or I for assistance.

From: Monsyne Dragon <mdragon@RACKSPACE.COM<mailto:mdragon@RACKSPACE.COM>>
Date: Mon, 23 Apr 2012 16:39:08 +0000
To: Luis Gervaso <luis@woorea.es<mailto:luis@woorea.es>>
Cc: "<openstack@lists.launchpad.net<mailto:openstack@lists.launchpad.net>>" <openstack@lists.launchpad.net<mailto:openstack@lists.launchpad.net>>
Subject: Re: [Openstack] Monitoring / Billing Architecture proposed

This already exists in trunk. The Notification system was designed specifically to feed billing and monitoring systems.

Basically, we don't want Nova/Glance/etc to be in the business of trying to determine billing logic, since it is different for pretty much everyone, so we just emit notifications to a queue and the interested pull what they want, and aggregate according to their own rules.

On Apr 22, 2012, at 1:50 PM, Luis Gervaso wrote:

Hi,

I want to share the architecture i am developing in order to perform the monitorig / billing OpenStack support:

1. AMQP Client which listen to RabbitMQ / QPid (this should be interchangeable) (Own Stuff or ServiceMix / Camel)

2. Events should be stored on a NoSQL document oriented database (I think mongodb is perfect, since we can query in a super easy fashion)

We have an existing system called Yagi (https://github.com/Cerberus98/yagi/) that listens to the notification queues and persists events to a Redis database. It then provides feeds as ATOM formatted documents that a billing system can pull to aggregate data, It also can support PubSub notification of clients thru the pubsubhubub protocol, and push events to a long-term archiving store thru the AtomPub protocol.

That said, the notification system outputs its events as JSON, so it should be easy to pipe into a json document-oriented db if that's what you need. (we only use ATOM because we have a atom-based archiving/search/aggregation engine (it's open source: http://atomhopper.org/ ) our in-house systems already plug into. )




3a. The monitoring system can pull/push MongoDB

3b. The billing system can pull to create invoices

4. A mediation EIP should be necessary to integrate a billing/monitoring product. (ServiceMix / Camel)

This is to receive your feedback. So please, critics are welcome!

Cheers!

--
-------------------------------------------
Luis Alberto Gervaso Martin
Woorea Solutions, S.L
CEO & CTO
mobile: (+34) 627983344
luis@<mailto:luis.gervaso@gmail.com>woorea.es<http://woorea.es/>


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net<mailto:openstack@lists.launchpad.net>
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

--
Monsyne M. Dragon
OpenStack/Nova
cell 210-441-0965
work x 5014190

_______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net<mailto:openstack@lists.launchpad.net> Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp

So, we could build on this. No reason to reinvent, but we might want to expand the number of events. I'm concerned about things like what happens when flavors change over time. Maybe the answer is, always append to the flavor/instance-type table. The code I remember and the admin interface that Ken wrote allowed you to modify flavors. That would break billing unless you also track flavor modifications.

-------------------------------------------------
Brian Schott, CTO
Nimbis Services, Inc.
brian.schott@nimbisservices.com
ph: 443-274-6064 fx: 443-274-6060



On Apr 23, 2012, at 1:40 PM, Luis Gervaso wrote:

> I have been looking at : http://wiki.openstack.org/SystemUsageData
>
> On Mon, Apr 23, 2012 at 7:35 PM, Brian Schott <brian.schott@nimbisservices.com> wrote:
> Is there a document somewhere on what events the services emit?
>
> -------------------------------------------------
> Brian Schott, CTO
> Nimbis Services, Inc.
> brian.schott@nimbisservices.com
> ph: 443-274-6064 fx: 443-274-6060
>
>
>
> On Apr 23, 2012, at 12:39 PM, Monsyne Dragon wrote:
>
>> This already exists in trunk. The Notification system was designed specifically to feed billing and monitoring systems.
>>
>> Basically, we don't want Nova/Glance/etc to be in the business of trying to determine billing logic, since it is different for pretty much everyone, so we just emit notifications to a queue and the interested pull what they want, and aggregate according to their own rules.
>>
>> On Apr 22, 2012, at 1:50 PM, Luis Gervaso wrote:
>>
>>> Hi,
>>>
>>> I want to share the architecture i am developing in order to perform the monitorig / billing OpenStack support:
>>>
>>> 1. AMQP Client which listen to RabbitMQ / QPid (this should be interchangeable) (Own Stuff or ServiceMix / Camel)
>>
>>> 2. Events should be stored on a NoSQL document oriented database (I think mongodb is perfect, since we can query in a super easy fashion)
>>
>> We have an existing system called Yagi (https://github.com/Cerberus98/yagi/) that listens to the notification queues and persists events to a Redis database. It then provides feeds as ATOM formatted documents that a billing system can pull to aggregate data, It also can support PubSub notification of clients thru the pubsubhubub protocol, and push events to a long-term archiving store thru the AtomPub protocol.
>>
>> That said, the notification system outputs its events as JSON, so it should be easy to pipe into a json document-oriented db if that's what you need. (we only use ATOM because we have a atom-based archiving/search/aggregation engine (it's open source: http://atomhopper.org/ ) our in-house systems already plug into. )
>>
>>
>>
>>>
>>> 3a. The monitoring system can pull/push MongoDB
>>>
>>> 3b. The billing system can pull to create invoices
>>>
>>> 4. A mediation EIP should be necessary to integrate a billing/monitoring product. (ServiceMix / Camel)
>>>
>>> This is to receive your feedback. So please, critics are welcome!
>>>
>>> Cheers!
>>>
>>> --
>>> -------------------------------------------
>>> Luis Alberto Gervaso Martin
>>> Woorea Solutions, S.L
>>> CEO & CTO
>>> mobile: (+34) 627983344
>>> luis@woorea.es
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to : openstack@lists.launchpad.net
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help : https://help.launchpad.net/ListHelp
>>
>> --
>> Monsyne M. Dragon
>> OpenStack/Nova
>> cell 210-441-0965
>> work x 5014190
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help : https://help.launchpad.net/ListHelp
>
>
>
>
> --
> -------------------------------------------
> Luis Alberto Gervaso Martin
> Woorea Solutions, S.L
> CEO & CTO
> mobile: (+34) 627983344
> luis@woorea.es
>

On 04/23/2012 07:06 PM, Luis Gervaso wrote:
> Nick, I want contribute in the discussion group.

Luis,
I just sent your an invite to the doodle to pick the best time for this
irc meeting. Once the date will have been finalized (I'll close the
poll tomorrow EOD), I'll announce the date, time and place on this list
as well as to everyone that entered the poll.

Anyone else wanting to participate in the poll for the meeting time may
follow this link:
http://doodle.com/zi6b6vxxkiuxyuqk

Thanks,
Nick

Brian,

Dough isn't dependent on Horizon.

Dough has a client that can be inserted anywhere to notify the start/end of
using a resource.

We inserted the client in horizon just to try out the many ways our billing
system could be integrated with our existing deployment.

We are planning to make a Nova notifier to listen on the AMQP to automate
the subscription process.

However, we still have to devise a way to integrate Swift that would
be symmetric with the things done in Nova.

Also fail recovery is a feature we are working on right now. Separate the
log and execution to cross-reference the logs and DB periodically.

We plan to finish the first release this month and use it on our deployment.

Let me know if you have other suggestions.

Cheers,
LZY

On Tue, Apr 24, 2012 at 1:06 AM, Doug Hellmann
<doug.hellmann@dreamhost.com>wrote:

>
>
> On Sun, Apr 22, 2012 at 5:29 PM, Brian Schott <
> brian.schott@nimbisservices.com> wrote:
>
>> Dough is a proposed billing service. There was a session at Folsom
>> design summit. This is a practical project for an OpenStack provider with
>> test code on github.
>> http://summit.openstack.org/sessions/view/37
>>
>> "
>> The billing system consists of three components.
>> 1) API Server, which receives subscribe/unsubscribe and usage query
>> requests.
>> 2) Farmer, which periodically checks all subscriptions and requests
>> billing to the collector.
>> 3) Collector, gets work from the farmer and uses python-novaclient,
>> kanyun-client, swift-client to retrieve usage information of a product the
>> tenant has subscribed and charges money to the accordingly.
>> "
>> Kanyun is an OpenStack monitoring tool, but I don't see documentation in
>> the code tree (but I've just pulled it to see).
>> https://github.com/lzyeval
>>
>> Dough had mixed reviews during the session, but that I think was because
>> it came across as a billing system triggered solely by Horizon requests and
>> was a polled architecture.
>>
>
> I also had some reservations about Dough because it seemed brittle. For
> example, it was my impression that if an instance creation operation failed
> for some reason, the operator had to manually correct the billing data.
>
>
>> Given what we have today, I'd implement my own billing/metering system
>> exactly the same way. In fact, I have. Our own e-commerce system at
>> Nimbis works this way with EC2 and OpenStack presently because the only
>> option available is polling periodically and logging the usage data.
>>
>
> The fact that it only worked through Horizon was the biggest issue for us.
> That means if we expose the API to our users, instances created via the API
> but not the Horizon dashboard would not be billed. I understood the
> decision to be a proof-of-concept, but an event-based model for collecting
> the data seems much more robust and flexible.
>
>
>> Where I'd like to see OpenStack go is metering service that is fully
>> asynchronous and event driven so that I only need to hit the API service
>> when I'm generating an invoice, rather than once per minute because I don't
>> know when an instance was terminated by the user or just crashed.
>>
>> -------------------------------------------------
>> Brian Schott, CTO
>> Nimbis Services, Inc.
>> brian.schott@nimbisservices.com
>> ph: 443-274-6064 fx: 443-274-6060
>>
>>
>>
>> On Apr 22, 2012, at 4:57 PM, Endre Karlson wrote:
>>
>> What is Dough then compared to what you want to do ?
>>
>> 2012/4/22 Endre Karlson <endre.karlson@gmail.com>
>>
>>> What is Dough then ?
>>>
>>>
>>> 2012/4/22 Brian Schott <brian.schott@nimbisservices.com>
>>>
>>>> I see this blueprint for metering, but none for Dough currently.
>>>> http://wiki.openstack.org/EfficientMetering
>>>>
>>>> Here are the Dough slides, however:
>>>> http://www.slideshare.net/lzyeval/dough-openstack-billing-project
>>>>
>>>> We collectively need to talk more about the user scenarios, because I
>>>> don't think you can just put a decorator around the API rpc calls and get
>>>> an accurate picture of what to bill for later. There are metered things
>>>> like bandwidth or IOPS, events that happen outside of the API (shutdown
>>>> -h), and it is hard to predict what a reseller will want to charge for. It
>>>> is better to put a metering system in that can handle many billing
>>>> configurations.
>>>>
>>>>
>>>> -------------------------------------------------
>>>> Brian Schott, CTO
>>>> Nimbis Services, Inc.
>>>> brian.schott@nimbisservices.com
>>>> ph: 443-274-6064 fx: 443-274-6060
>>>>
>>>>
>>>>
>>>> On Apr 22, 2012, at 3:36 PM, Luis Gervaso wrote:
>>>>
>>>> Dough is the proposed billing platform/product (where the billing rules
>>>> live), isn't it?
>>>>
>>>> I don't know Dough enough, so please me correct me if i'm wrong.
>>>>
>>>> I'm trying to define a generic/agnostic integration process, obviously
>>>> where Dough
>>>> can fit perfectly. I would like it become part to the reference
>>>> architecture.
>>>>
>>>> Option 1) [3b in the arch proposed]
>>>>
>>>> Dough should pull NoSQL
>>>>
>>>> Option 2)
>>>>
>>>> A Mediator can feed Dough
>>>>
>>>>
>>>> On Sun, Apr 22, 2012 at 9:13 PM, Endre Karlson <endre.karlson@gmail.com
>>>> > wrote:
>>>>
>>>>> What about using the Dough project?
>>>>>
>>>>> Endre.
>>>>>
>>>>>
>>>>> 2012/4/22 Endre Karlson <endre.karlson@gmail.com>
>>>>>
>>>>>> What about using the Dough project ?
>>>>>>
>>>>>> Endre.
>>>>>>
>>>>>> 2012/4/22 Luis Gervaso <luis@woorea.es>
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I want to share the architecture i am developing in order to perform
>>>>>>> the monitorig / billing OpenStack support:
>>>>>>>
>>>>>>> 1. AMQP Client which listen to RabbitMQ / QPid (this should be
>>>>>>> interchangeable) (Own Stuff or ServiceMix / Camel)
>>>>>>>
>>>>>>> 2. Events should be stored on a NoSQL document oriented database (I
>>>>>>> think mongodb is perfect, since we can query in a super easy fashion)
>>>>>>>
>>>>>>> 3a. The monitoring system can pull/push MongoDB
>>>>>>>
>>>>>>> 3b. The billing system can pull to create invoices
>>>>>>>
>>>>>>> 4. A mediation EIP should be necessary to integrate a
>>>>>>> billing/monitoring product. (ServiceMix / Camel)
>>>>>>>
>>>>>>> This is to receive your feedback. So please, critics are welcome!
>>>>>>>
>>>>>>> Cheers!
>>>>>>>
>>>>>>> --
>>>>>>> -------------------------------------------
>>>>>>> Luis Alberto Gervaso Martin
>>>>>>> Woorea Solutions, S.L
>>>>>>> CEO & CTO
>>>>>>> mobile: (+34) 627983344
>>>>>>> luis@ <luis.gervaso@gmail.com>woorea.es
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Mailing list: https://launchpad.net/~openstack
>>>>>>> Post to : openstack@lists.launchpad.net
>>>>>>> Unsubscribe : https://launchpad.net/~openstack
>>>>>>> More help : https://help.launchpad.net/ListHelp
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Mailing list: https://launchpad.net/~openstack
>>>>> Post to : openstack@lists.launchpad.net
>>>>> Unsubscribe : https://launchpad.net/~openstack
>>>>> More help : https://help.launchpad.net/ListHelp
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> -------------------------------------------
>>>> Luis Alberto Gervaso Martin
>>>> Woorea Solutions, S.L
>>>> CEO & CTO
>>>> mobile: (+34) 627983344
>>>> luis@ <luis.gervaso@gmail.com>woorea.es
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~openstack
>>>> Post to : openstack@lists.launchpad.net
>>>> Unsubscribe : https://launchpad.net/~openstack
>>>> More help : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>>
>>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help : https://help.launchpad.net/ListHelp
>>
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help : https://help.launchpad.net/ListHelp
>>
>>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
>
>

I like http://wiki.openstack.org/SystemUsageData

I just wonder if there should be offical formats for the data sent across the wire.

Or at least high level formats that define the data, maybe in a simple python "map" style layout.

With examples would be awesome.

On 4/23/12 10:40 AM, "Luis Gervaso" <luis@woorea.es> wrote:

I have been looking at : http://wiki.openstack.org/SystemUsageData

On Mon, Apr 23, 2012 at 7:35 PM, Brian Schott <brian.schott@nimbisservices.com> wrote:
Is there a document somewhere on what events the services emit?

-------------------------------------------------
Brian Schott, CTO
Nimbis Services, Inc.
brian.schott@nimbisservices.com
ph: 443-274-6064 <tel:443-274-6064> fx: 443-274-6060 <tel:443-274-6060>



On Apr 23, 2012, at 12:39 PM, Monsyne Dragon wrote:

This already exists in trunk. The Notification system was designed specifically to feed billing and monitoring systems.

Basically, we don't want Nova/Glance/etc to be in the business of trying to determine billing logic, since it is different for pretty much everyone, so we just emit notifications to a queue and the interested pull what they want, and aggregate according to their own rules.

On Apr 22, 2012, at 1:50 PM, Luis Gervaso wrote:

Hi,

I want to share the architecture i am developing in order to perform the monitorig / billing OpenStack support:

1. AMQP Client which listen to RabbitMQ / QPid (this should be interchangeable) (Own Stuff or ServiceMix / Camel)

2. Events should be stored on a NoSQL document oriented database (I think mongodb is perfect, since we can query in a super easy fashion)

We have an existing system called Yagi (https://github.com/Cerberus98/yagi/) that listens to the notification queues and persists events to a Redis database. It then provides feeds as ATOM formatted documents that a billing system can pull to aggregate data, It also can support PubSub notification of clients thru the pubsubhubub protocol, and push events to a long-term archiving store thru the AtomPub protocol.

That said, the notification system outputs its events as JSON, so it should be easy to pipe into a json document-oriented db if that's what you need. (we only use ATOM because we have a atom-based archiving/search/aggregation engine (it's open source: http://atomhopper.org/ ) our in-house systems already plug into. )




3a. The monitoring system can pull/push MongoDB

3b. The billing system can pull to create invoices

4. A mediation EIP should be necessary to integrate a billing/monitoring product. (ServiceMix / Camel)

This is to receive your feedback. So please, critics are welcome!

Cheers!

On Mon, Apr 23, 2012 at 1:50 PM, Brian Schott <
brian.schott@nimbisservices.com> wrote:

> So, we could build on this. No reason to reinvent, but we might want to
> expand the number of events. I'm concerned about things like what happens
> when flavors change over time. Maybe the answer is, always append to the
> flavor/instance-type table. The code I remember and the admin interface
> that Ken wrote allowed you to modify flavors. That would break billing
> unless you also track flavor modifications.
>

That seems like a situation where you would want to denormalize the billing
database and record the flavor details along with the rest of the creation
event data.

Doug

Joshua,

I have performed a create instance operation and here is an example data
obtained from stable/essex rabbitmq nova catch all exchange.

[*] Waiting for messages. To exit press CTRL+C

[x] Received '{"_context_roles": ["admin"], "_msg_id":
"a2d13735baad4613b89c6132e0fa8302", "_context_read_deleted": "no",
"_context_request_id": "req-d7ffbe78-7a9c-4d20-9ac5-3e56951526fe", "args":
{"instance_id": 6, "instance_uuid": "e3ad17e6-dd59-4b67-a7d0-e3812f96c2d7",
"host": "ubuntu", "project_id": "c290118b14564257be26a2cb901721a2",
"rxtx_factor": 1.0}, "_context_auth_token": null, "_context_is_admin":
true, "_context_project_id": null, "_context_timestamp":
"2012-03-24T01:36:48.774891", "_context_user_id": null, "method":
"get_instance_nw_info", "_context_remote_address": null}'

[x] Received '{"_context_roles": ["admin"], "_msg_id":
"a1cb1cf61e5441c2a772b29d3cd54202", "_context_read_deleted": "no",
"_context_request_id": "req-db34ba32-8bd9-4cd5-b7b5-43705a9e258e", "args":
{"instance_id": 6, "instance_uuid": "e3ad17e6-dd59-4b67-a7d0-e3812f96c2d7",
"host": "ubuntu", "project_id": "c290118b14564257be26a2cb901721a2",
"rxtx_factor": 1.0}, "_context_auth_token": null, "_context_is_admin":
true, "_context_project_id": null, "_context_timestamp":
"2012-03-24T01:37:50.463586", "_context_user_id": null, "method":
"get_instance_nw_info", "_context_remote_address": null}'

[x] Received '{"_context_roles": ["admin"], "_msg_id":
"ebb0b1c340de4024a22eafec9d0a2d66", "_context_read_deleted": "no",
"_context_request_id": "req-ddb51b2b-a29f-4aad-909d-3f7f79f053c4", "args":
{"instance_id": 6, "instance_uuid": "e3ad17e6-dd59-4b67-a7d0-e3812f96c2d7",
"host": "ubuntu", "project_id": "c290118b14564257be26a2cb901721a2",
"rxtx_factor": 1.0}, "_context_auth_token": null, "_context_is_admin":
true, "_context_project_id": null, "_context_timestamp":
"2012-03-24T01:38:59.217333", "_context_user_id": null, "method":
"get_instance_nw_info", "_context_remote_address": null}'

[x] Received '{"_context_roles": ["Member"], "_msg_id":
"729535c00d224414a98286e9ce3475a9", "_context_read_deleted": "no",
"_context_request_id": "req-b056a8cc-3542-41a9-9e58-8fb592086264",
"_context_auth_token": "deb477655fba448e85199f7e559da77a",
"_context_is_admin": false, "_context_project_id":
"df3827f76f714b1e8f31675caf84ae9d", "_context_timestamp":
"2012-03-24T01:39:19.813393", "_context_user_id":
"abe21eb7e6884547810f0a43c216e6a6", "method":
"get_floating_ips_by_project", "_context_remote_address": "192.168.1.41"}'

[x] Received '{"_context_roles": ["Member", "admin"],
"_context_request_id": "req-45e6c2af-52c7-4de3-af6c-6b2f7520cfd5",
"_context_read_deleted": "no", "args": {"request_spec": {"num_instances":
1, "block_device_mapping": [], "image": {"status": "active", "name":
"cirros-0.3.0-x86_64-uec", "deleted": false, "container_format": "ami",
"created_at": "2012-03-20 17:37:08", "disk_format": "ami", "updated_at":
"2012-03-20 17:37:08", "properties": {"kernel_id":
"6b700d25-3293-420a-82e4-8247d4b0da2a", "ramdisk_id":
"22b10c35-c868-4470-84ef-54ae9f17a977"}, "min_ram": "0", "checksum":
"2f81976cae15c16ef0010c51e3a6c163", "min_disk": "0", "is_public": true,
"deleted_at": null, "id": "f7d4bea2-2aed-4bf3-a5cb-db6a34c4a525", "size":
25165824}, "instance_type": {"root_gb": 0, "name": "m1.tiny", "deleted":
false, "created_at": null, "ephemeral_gb": 0, "updated_at": null,
"memory_mb": 512, "vcpus": 1, "flavorid": "1", "swap": 0, "rxtx_factor":
1.0, "extra_specs": {}, "deleted_at": null, "vcpu_weight": null, "id": 2},
"instance_properties": {"vm_state": "building", "ephemeral_gb": 0,
"access_ip_v6": null, "access_ip_v4": null, "kernel_id":
"6b700d25-3293-420a-82e4-8247d4b0da2a", "key_name": "testssh",
"ramdisk_id": "22b10c35-c868-4470-84ef-54ae9f17a977", "instance_type_id":
2, "user_data": "dGhpcyBpcyBteSB1c2VyIGRhdGE=", "vm_mode": null,
"display_name": "eureka", "config_drive_id": "", "reservation_id":
"r-xtzjx50j", "key_data": "ssh-rsa
AAAAB3NzaC1yc2EAAAADAQABAAAAgQDJ31tdayh1xnAY+JO/ZVdg5L83CsIU7qaOmFubdH7zlg2jjS9JmkPNANj99zx+UHg5F5JKGMef9M8VP/V89D5g0oIjIJtBdFpKOScBo3yJ1vteW5ItImH8h9TldymHf+CWNVY1oNNqzXqAb41xwUUDNvgeXHRZNnE6tmwZO0oC1Q==
stack@ubuntu\n", "root_gb": 0, "user_id":
"abe21eb7e6884547810f0a43c216e6a6", "uuid":
"40b5a1c5-bd4f-40ee-ae0a-73e0bc927431", "root_device_name": null,
"availability_zone": null, "launch_time": "2012-03-24T01:39:52Z",
"metadata": {}, "display_description": "eureka", "memory_mb": 512,
"launch_index": 0, "vcpus": 1, "locked": false, "image_ref":
"f7d4bea2-2aed-4bf3-a5cb-db6a34c4a525", "architecture": null,
"power_state": 0, "auto_disk_config": null, "progress": 0, "os_type": null,
"project_id": "df3827f76f714b1e8f31675caf84ae9d", "config_drive": ""},
"security_group": ["default"]}, "is_first_time": true, "filter_properties":
{"scheduler_hints": {}}, "topic": "compute", "admin_password":
"SKohh79r956J", "injected_files": [], "requested_networks": null},
"_context_auth_token": "deb477655fba448e85199f7e559da77a",
"_context_is_admin": false, "_context_project_id":
"df3827f76f714b1e8f31675caf84ae9d", "_context_timestamp":
"2012-03-24T01:39:52.089383", "_context_user_id":
"abe21eb7e6884547810f0a43c216e6a6", "method": "run_instance",
"_context_remote_address": "192.168.1.41"}'

[x] Received '{"_context_roles": ["Member", "admin"],
"_context_request_id": "req-45e6c2af-52c7-4de3-af6c-6b2f7520cfd5",
"_context_read_deleted": "no", "args": {"instance_uuid":
"40b5a1c5-bd4f-40ee-ae0a-73e0bc927431", "requested_networks": null,
"is_first_time": true, "admin_password": "SKohh79r956J", "injected_files":
[]}, "_context_auth_token": "deb477655fba448e85199f7e559da77a",
"_context_is_admin": true, "_context_project_id":
"df3827f76f714b1e8f31675caf84ae9d", "_context_timestamp":
"2012-03-24T01:39:52.089383", "_context_user_id":
"abe21eb7e6884547810f0a43c216e6a6", "method": "run_instance",
"_context_remote_address": "192.168.1.41"}'

[x] Received '{"_context_roles": ["Member", "admin"], "_msg_id":
"f40e21507437492f97a02cd25415498a", "_context_read_deleted": "no",
"_context_request_id": "req-45e6c2af-52c7-4de3-af6c-6b2f7520cfd5", "args":
{"instance_uuid": "40b5a1c5-bd4f-40ee-ae0a-73e0bc927431", "vpn": false,
"requested_networks": null, "instance_id": 7, "host": "ubuntu",
"rxtx_factor": 1.0, "project_id": "df3827f76f714b1e8f31675caf84ae9d"},
"_context_auth_token": "deb477655fba448e85199f7e559da77a",
"_context_is_admin": true, "_context_project_id":
"df3827f76f714b1e8f31675caf84ae9d", "_context_timestamp":
"2012-03-24T01:39:52.089383", "_context_user_id":
"abe21eb7e6884547810f0a43c216e6a6", "method": "allocate_for_instance",
"_context_remote_address": "192.168.1.41"}'

[x] Received '{"_context_roles": ["admin"], "_msg_id":
"96c3d16edf894a89ae85ed90b0a0858b", "_context_read_deleted": "no",
"_context_request_id": "req-81c9353b-f912-408e-a297-0e8ad6b8fe10", "args":
{"instance_id": 6, "instance_uuid": "e3ad17e6-dd59-4b67-a7d0-e3812f96c2d7",
"host": "ubuntu", "project_id": "c290118b14564257be26a2cb901721a2",
"rxtx_factor": 1.0}, "_context_auth_token": null, "_context_is_admin":
true, "_context_project_id": null, "_context_timestamp":
"2012-03-24T01:40:01.390757", "_context_user_id": null, "method":
"get_instance_nw_info", "_context_remote_address": null}'

[x] Received '{"_context_roles": ["admin"], "_context_request_id":
"req-d0707421-7f4e-4f1f-bf89-109ca4625ca5", "_context_read_deleted": "no",
"args": {"address": "10.0.0.2"}, "_context_auth_token": null,
"_context_is_admin": true, "_context_project_id": null,
"_context_timestamp": "2012-03-24T01:40:53.338021", "_context_user_id":
null, "method": "lease_fixed_ip", "_context_remote_address": null}'

[x] Received '{"_context_roles": ["admin"], "_msg_id":
"38ad50d1abf445118c60017ee03851f6", "_context_read_deleted": "no",
"_context_request_id": "req-51cd0d75-17e5-414b-affd-1ca2060cc8cb", "args":
{"instance_id": 7, "instance_uuid": "40b5a1c5-bd4f-40ee-ae0a-73e0bc927431",
"host": "ubuntu", "project_id": "df3827f76f714b1e8f31675caf84ae9d",
"rxtx_factor": 1.0}, "_context_auth_token": null, "_context_is_admin":
true, "_context_project_id": null, "_context_timestamp":
"2012-03-24T01:41:07.580157", "_context_user_id": null, "method":
"get_instance_nw_info", "_context_remote_address": null}'

On Mon, Apr 23, 2012 at 9:23 PM, Doug Hellmann
<doug.hellmann@dreamhost.com>wrote:

>
>
> On Mon, Apr 23, 2012 at 1:50 PM, Brian Schott <
> brian.schott@nimbisservices.com> wrote:
>
>> So, we could build on this. No reason to reinvent, but we might want to
>> expand the number of events. I'm concerned about things like what happens
>> when flavors change over time. Maybe the answer is, always append to the
>> flavor/instance-type table. The code I remember and the admin interface
>> that Ken wrote allowed you to modify flavors. That would break billing
>> unless you also track flavor modifications.
>>
>
> That seems like a situation where you would want to denormalize the
> billing database and record the flavor details along with the rest of the
> creation event data.
>
> Doug
>
>


--
-------------------------------------------
Luis Alberto Gervaso Martin
Woorea Solutions, S.L
CEO & CTO
mobile: (+34) 627983344
luis@ <luis.gervaso@gmail.com>woorea.es

The following links to work from OGF in the usage accounting and tracking area might be useful.

First of all, we have the UsageRecord format (UR), which has been used in a variety of distributed computing environments for tracking usage. It defines an XML-based format for exchange of usage records. The format is general enough to describe a variety of distributed computing environments, but specific enough to allow practical usage and has served as the basis for usage accounting in grids, for example, for some time.

http://ogf.org/documents/GFD.98.pdf

We also have the following storage accounting work in process and nearly complete:

https://forge.gridforum.org/sf/go/artf6532?nav=1 and
http://www.ogf.org/Public_Comment_Docs/Documents/2012-02/EMI-StAR-OGF-info-doc-v2.pdf

This document has just come out of public comment and is moving toward publication shortly.

Between the UR work on traditional cpu accounting and the above storage accounting publication, I hope this provides some input into this topic from OGF.

Alan

On Apr 23, 2012, at 12:50 PM, Brian Schott wrote:

> So, we could build on this. No reason to reinvent, but we might want to expand the number of events. I'm concerned about things like what happens when flavors change over time. Maybe the answer is, always append to the flavor/instance-type table. The code I remember and the admin interface that Ken wrote allowed you to modify flavors. That would break billing unless you also track flavor modifications.
>
> -------------------------------------------------
> Brian Schott, CTO
> Nimbis Services, Inc.
> brian.schott@nimbisservices.com
> ph: 443-274-6064 fx: 443-274-6060
>
>
>
> On Apr 23, 2012, at 1:40 PM, Luis Gervaso wrote:
>
>> I have been looking at : http://wiki.openstack.org/SystemUsageData
>>
>> On Mon, Apr 23, 2012 at 7:35 PM, Brian Schott <brian.schott@nimbisservices.com> wrote:
>> Is there a document somewhere on what events the services emit?
>>
>> -------------------------------------------------
>> Brian Schott, CTO
>> Nimbis Services, Inc.
>> brian.schott@nimbisservices.com
>> ph: 443-274-6064 fx: 443-274-6060
>>
>>
>>
>> On Apr 23, 2012, at 12:39 PM, Monsyne Dragon wrote:
>>
>>> This already exists in trunk. The Notification system was designed specifically to feed billing and monitoring systems.
>>>
>>> Basically, we don't want Nova/Glance/etc to be in the business of trying to determine billing logic, since it is different for pretty much everyone, so we just emit notifications to a queue and the interested pull what they want, and aggregate according to their own rules.
>>>
>>> On Apr 22, 2012, at 1:50 PM, Luis Gervaso wrote:
>>>
>>>> Hi,
>>>>
>>>> I want to share the architecture i am developing in order to perform the monitorig / billing OpenStack support:
>>>>
>>>> 1. AMQP Client which listen to RabbitMQ / QPid (this should be interchangeable) (Own Stuff or ServiceMix / Camel)
>>>
>>>> 2. Events should be stored on a NoSQL document oriented database (I think mongodb is perfect, since we can query in a super easy fashion)
>>>
>>> We have an existing system called Yagi (https://github.com/Cerberus98/yagi/) that listens to the notification queues and persists events to a Redis database. It then provides feeds as ATOM formatted documents that a billing system can pull to aggregate data, It also can support PubSub notification of clients thru the pubsubhubub protocol, and push events to a long-term archiving store thru the AtomPub protocol.
>>>
>>> That said, the notification system outputs its events as JSON, so it should be easy to pipe into a json document-oriented db if that's what you need. (we only use ATOM because we have a atom-based archiving/search/aggregation engine (it's open source: http://atomhopper.org/ ) our in-house systems already plug into. )
>>>
>>>
>>>
>>>>
>>>> 3a. The monitoring system can pull/push MongoDB
>>>>
>>>> 3b. The billing system can pull to create invoices
>>>>
>>>> 4. A mediation EIP should be necessary to integrate a billing/monitoring product. (ServiceMix / Camel)
>>>>
>>>> This is to receive your feedback. So please, critics are welcome!
>>>>
>>>> Cheers!
>>>>
>>>> --
>>>> -------------------------------------------
>>>> Luis Alberto Gervaso Martin
>>>> Woorea Solutions, S.L
>>>> CEO & CTO
>>>> mobile: (+34) 627983344
>>>> luis@woorea.es
>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~openstack
>>>> Post to : openstack@lists.launchpad.net
>>>> Unsubscribe : https://launchpad.net/~openstack
>>>> More help : https://help.launchpad.net/ListHelp
>>>
>>> --
>>> Monsyne M. Dragon
>>> OpenStack/Nova
>>> cell 210-441-0965
>>> work x 5014190
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to : openstack@lists.launchpad.net
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help : https://help.launchpad.net/ListHelp
>>
>>
>>
>>
>> --
>> -------------------------------------------
>> Luis Alberto Gervaso Martin
>> Woorea Solutions, S.L
>> CEO & CTO
>> mobile: (+34) 627983344
>> luis@woorea.es
>>
>
> <smime.p7s>_______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

Flavor information is copied to the Instance table on creation so the
Flavors can change and still be tracked in the Instance. It may just
need to be sent in the notification payload.

The current events in the system are documented here:
http://wiki.openstack.org/SystemUsageData

-Sandy


On 04/23/2012 02:50 PM, Brian Schott wrote:
> So, we could build on this. No reason to reinvent, but we might want to
> expand the number of events. I'm concerned about things like what
> happens when flavors change over time. Maybe the answer is, always
> append to the flavor/instance-type table. The code I remember and the
> admin interface that Ken wrote allowed you to modify flavors. That
> would break billing unless you also track flavor modifications.
>
> -------------------------------------------------
> Brian Schott, CTO
> Nimbis Services, Inc.
> brian.schott@nimbisservices.com <mailto:brian.schott@nimbisservices.com>
> ph: 443-274-6064 fx: 443-274-6060
>
>
>
> On Apr 23, 2012, at 1:40 PM, Luis Gervaso wrote:
>
>> I have been looking at : http://wiki.openstack.org/SystemUsageData
>>
>> On Mon, Apr 23, 2012 at 7:35 PM, Brian Schott
>> <brian.schott@nimbisservices.com
>> <mailto:brian.schott@nimbisservices.com>> wrote:
>>
>> Is there a document somewhere on what events the services emit?
>>
>> -------------------------------------------------
>> Brian Schott, CTO
>> Nimbis Services, Inc.
>> brian.schott@nimbisservices.com
>> <mailto:brian.schott@nimbisservices.com>
>> ph: 443-274-6064 <tel:443-274-6064> fx: 443-274-6060
>> <tel:443-274-6060>
>>
>>
>>
>> On Apr 23, 2012, at 12:39 PM, Monsyne Dragon wrote:
>>
>>> This already exists in trunk. The Notification system was
>>> designed specifically to feed billing and monitoring systems.
>>>
>>> Basically, we don't want Nova/Glance/etc to be in the business of
>>> trying to determine billing logic, since it is different for
>>> pretty much everyone, so we just emit notifications to a queue
>>> and the interested pull what they want, and aggregate according
>>> to their own rules.
>>>
>>> On Apr 22, 2012, at 1:50 PM, Luis Gervaso wrote:
>>>
>>>> Hi,
>>>>
>>>> I want to share the architecture i am developing in order to
>>>> perform the monitorig / billing OpenStack support:
>>>>
>>>> 1. AMQP Client which listen to RabbitMQ / QPid (this should be
>>>> interchangeable) (Own Stuff or ServiceMix / Camel)
>>>
>>>> 2. Events should be stored on a NoSQL document oriented database
>>>> (I think mongodb is perfect, since we can query in a super easy
>>>> fashion)
>>>
>>> We have an existing system called Yagi
>>> (https://github.com/Cerberus98/yagi/) that listens to the
>>> notification queues and persists events to a Redis database. It
>>> then provides feeds as ATOM formatted documents that a billing
>>> system can pull to aggregate data, It also can support PubSub
>>> notification of clients thru the pubsubhubub protocol, and push
>>> events to a long-term archiving store thru the AtomPub protocol.
>>>
>>> That said, the notification system outputs its events as JSON, so
>>> it should be easy to pipe into a json document-oriented db if
>>> that's what you need. (we only use ATOM because we have a
>>> atom-based archiving/search/aggregation engine (it's open
>>> source: http://atomhopper.org/ ) our in-house systems already
>>> plug into. )
>>>
>>>
>>>
>>>>
>>>> 3a. The monitoring system can pull/push MongoDB
>>>>
>>>> 3b. The billing system can pull to create invoices
>>>>
>>>> 4. A mediation EIP should be necessary to integrate a
>>>> billing/monitoring product. (ServiceMix / Camel)
>>>>
>>>> This is to receive your feedback. So please, critics are welcome!
>>>>
>>>> Cheers!
>>>>
>>>> --
>>>> -------------------------------------------
>>>> Luis Alberto Gervaso Martin
>>>> Woorea Solutions, S.L
>>>> CEO & CTO
>>>> mobile: (+34) 627983344 <tel:%28%2B34%29%20627983344>
>>>> luis@ <mailto:luis.gervaso@gmail.com>woorea.es <http://woorea.es/>
>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~openstack
>>>> Post to : openstack@lists.launchpad.net
>>>> <mailto:openstack@lists.launchpad.net>
>>>> Unsubscribe : https://launchpad.net/~openstack
>>>> More help : https://help.launchpad.net/ListHelp
>>>
>>> --
>>> Monsyne M. Dragon
>>> OpenStack/Nova
>>> cell 210-441-0965 <tel:210-441-0965>
>>> work x 5014190
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to : openstack@lists.launchpad.net
>>> <mailto:openstack@lists.launchpad.net>
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help : https://help.launchpad.net/ListHelp
>>
>>
>>
>>
>> --
>> -------------------------------------------
>> Luis Alberto Gervaso Martin
>> Woorea Solutions, S.L
>> CEO & CTO
>> mobile: (+34) 627983344
>> luis@ <mailto:luis.gervaso@gmail.com>woorea.es <http://woorea.es/>
>>
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

Doug,

Do we mirror the table structure of nova, etc. and add created/modified columns?

Or do we flatten into an instance event record with everything?

Bran

-------------------------------------------------
Brian Schott, CTO
Nimbis Services, Inc.
brian.schott@nimbisservices.com
ph: 443-274-6064 fx: 443-274-6060



On Apr 23, 2012, at 3:23 PM, Doug Hellmann wrote:

>
>
> On Mon, Apr 23, 2012 at 1:50 PM, Brian Schott <brian.schott@nimbisservices.com> wrote:
> So, we could build on this. No reason to reinvent, but we might want to expand the number of events. I'm concerned about things like what happens when flavors change over time. Maybe the answer is, always append to the flavor/instance-type table. The code I remember and the admin interface that Ken wrote allowed you to modify flavors. That would break billing unless you also track flavor modifications.
>
> That seems like a situation where you would want to denormalize the billing database and record the flavor details along with the rest of the creation event data.
>
> Doug
>

On Mon, Apr 23, 2012 at 4:14 PM, Brian Schott <
brian.schott@nimbisservices.com> wrote:

> Doug,
>
> Do we mirror the table structure of nova, etc. and add created/modified
> columns?
>

> Or do we flatten into an instance event record with everything?
>

I lean towards flattening the data as it is recorded and making a second
pass during the bill calculation. You need to record instance modifications
separately from the creation, especially if the modification changes the
billing rate. So you might have records for:

created instance, with UUID, name, size, timestamp, ownership information,
etc.
resized instance, with UUID, name, new size, timestamp, ownership
information, etc.
deleted instance, with UUID, name, size, timestamp, ownership information,
etc.

Maybe some of those values don't need to be reported in some cases, but if
you record a complete picture of the state of the instance then the code
that aggregates the event records to produce billing information can use it
to make decisions about how to record the charges.

There is also the case where an instance is still no longer running but
nova thinks it is (or the reverse), so some sort of auditing sweep needs to
be included (I think that's what Dough called the "farmer" but I don't have
my notes in front of me).


>
> Bran
>
> -------------------------------------------------
> Brian Schott, CTO
> Nimbis Services, Inc.
> brian.schott@nimbisservices.com
> ph: 443-274-6064 fx: 443-274-6060
>
>
>
> On Apr 23, 2012, at 3:23 PM, Doug Hellmann wrote:
>
>
>
> On Mon, Apr 23, 2012 at 1:50 PM, Brian Schott <
> brian.schott@nimbisservices.com> wrote:
>
>> So, we could build on this. No reason to reinvent, but we might want to
>> expand the number of events. I'm concerned about things like what happens
>> when flavors change over time. Maybe the answer is, always append to the
>> flavor/instance-type table. The code I remember and the admin interface
>> that Ken wrote allowed you to modify flavors. That would break billing
>> unless you also track flavor modifications.
>>
>
> That seems like a situation where you would want to denormalize the
> billing database and record the flavor details along with the rest of the
> creation event data.
>
> Doug
>
>
>

Great,

Now we just need to "officialize" those as a first step to making a real interchange format that can be versioned, documented... and all the other goodies u would normally expect.

On 4/23/12 12:26 PM, "Luis Gervaso" <luis@woorea.es> wrote:

Joshua,

I have performed a create instance operation and here is an example data obtained from stable/essex rabbitmq nova catch all exchange.
[*] Waiting for messages. To exit press CTRL+C

[x] Received '{"_context_roles": ["admin"], "_msg_id": "a2d13735baad4613b89c6132e0fa8302", "_context_read_deleted": "no", "_context_request_id": "req-d7ffbe78-7a9c-4d20-9ac5-3e56951526fe", "args": {"instance_id": 6, "instance_uuid": "e3ad17e6-dd59-4b67-a7d0-e3812f96c2d7", "host": "ubuntu", "project_id": "c290118b14564257be26a2cb901721a2", "rxtx_factor": 1.0}, "_context_auth_token": null, "_context_is_admin": true, "_context_project_id": null, "_context_timestamp": "2012-03-24T01:36:48.774891", "_context_user_id": null, "method": "get_instance_nw_info", "_context_remote_address": null}'

[x] Received '{"_context_roles": ["admin"], "_msg_id": "a1cb1cf61e5441c2a772b29d3cd54202", "_context_read_deleted": "no", "_context_request_id": "req-db34ba32-8bd9-4cd5-b7b5-43705a9e258e", "args": {"instance_id": 6, "instance_uuid": "e3ad17e6-dd59-4b67-a7d0-e3812f96c2d7", "host": "ubuntu", "project_id": "c290118b14564257be26a2cb901721a2", "rxtx_factor": 1.0}, "_context_auth_token": null, "_context_is_admin": true, "_context_project_id": null, "_context_timestamp": "2012-03-24T01:37:50.463586", "_context_user_id": null, "method": "get_instance_nw_info", "_context_remote_address": null}'

[x] Received '{"_context_roles": ["admin"], "_msg_id": "ebb0b1c340de4024a22eafec9d0a2d66", "_context_read_deleted": "no", "_context_request_id": "req-ddb51b2b-a29f-4aad-909d-3f7f79f053c4", "args": {"instance_id": 6, "instance_uuid": "e3ad17e6-dd59-4b67-a7d0-e3812f96c2d7", "host": "ubuntu", "project_id": "c290118b14564257be26a2cb901721a2", "rxtx_factor": 1.0}, "_context_auth_token": null, "_context_is_admin": true, "_context_project_id": null, "_context_timestamp": "2012-03-24T01:38:59.217333", "_context_user_id": null, "method": "get_instance_nw_info", "_context_remote_address": null}'

[x] Received '{"_context_roles": ["Member"], "_msg_id": "729535c00d224414a98286e9ce3475a9", "_context_read_deleted": "no", "_context_request_id": "req-b056a8cc-3542-41a9-9e58-8fb592086264", "_context_auth_token": "deb477655fba448e85199f7e559da77a", "_context_is_admin": false, "_context_project_id": "df3827f76f714b1e8f31675caf84ae9d", "_context_timestamp": "2012-03-24T01:39:19.813393", "_context_user_id": "abe21eb7e6884547810f0a43c216e6a6", "method": "get_floating_ips_by_project", "_context_remote_address": "192.168.1.41"}'

[x] Received '{"_context_roles": ["Member", "admin"], "_context_request_id": "req-45e6c2af-52c7-4de3-af6c-6b2f7520cfd5", "_context_read_deleted": "no", "args": {"request_spec": {"num_instances": 1, "block_device_mapping": [], "image": {"status": "active", "name": "cirros-0.3.0-x86_64-uec", "deleted": false, "container_format": "ami", "created_at": "2012-03-20 17:37:08", "disk_format": "ami", "updated_at": "2012-03-20 17:37:08", "properties": {"kernel_id": "6b700d25-3293-420a-82e4-8247d4b0da2a", "ramdisk_id": "22b10c35-c868-4470-84ef-54ae9f17a977"}, "min_ram": "0", "checksum": "2f81976cae15c16ef0010c51e3a6c163", "min_disk": "0", "is_public": true, "deleted_at": null, "id": "f7d4bea2-2aed-4bf3-a5cb-db6a34c4a525", "size": 25165824}, "instance_type": {"root_gb": 0, "name": "m1.tiny", "deleted": false, "created_at": null, "ephemeral_gb": 0, "updated_at": null, "memory_mb": 512, "vcpus": 1, "flavorid": "1", "swap": 0, "rxtx_factor": 1.0, "extra_specs": {}, "deleted_at": null, "vcpu_weight": null, "id": 2}, "instance_properties": {"vm_state": "building", "ephemeral_gb": 0, "access_ip_v6": null, "access_ip_v4": null, "kernel_id": "6b700d25-3293-420a-82e4-8247d4b0da2a", "key_name": "testssh", "ramdisk_id": "22b10c35-c868-4470-84ef-54ae9f17a977", "instance_type_id": 2, "user_data": "dGhpcyBpcyBteSB1c2VyIGRhdGE=", "vm_mode": null, "display_name": "eureka", "config_drive_id": "", "reservation_id": "r-xtzjx50j", "key_data": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAgQDJ31tdayh1xnAY+JO/ZVdg5L83CsIU7qaOmFubdH7zlg2jjS9JmkPNANj99zx+UHg5F5JKGMef9M8VP/V89D5g0oIjIJtBdFpKOScBo3yJ1vteW5ItImH8h9TldymHf+CWNVY1oNNqzXqAb41xwUUDNvgeXHRZNnE6tmwZO0oC1Q== stack@ubuntu\n", "root_gb": 0, "user_id": "abe21eb7e6884547810f0a43c216e6a6", "uuid": "40b5a1c5-bd4f-40ee-ae0a-73e0bc927431", "root_device_name": null, "availability_zone": null, "launch_time": "2012-03-24T01:39:52Z", "metadata": {}, "display_description": "eureka", "memory_mb": 512, "launch_index": 0, "vcpus": 1, "locked": false, "image_ref": "f7d4bea2-2aed-4bf3-a5cb-db6a34c4a525", "architecture": null, "power_state": 0, "auto_disk_config": null, "progress": 0, "os_type": null, "project_id": "df3827f76f714b1e8f31675caf84ae9d", "config_drive": ""}, "security_group": ["default"]}, "is_first_time": true, "filter_properties": {"scheduler_hints": {}}, "topic": "compute", "admin_password": "SKohh79r956J", "injected_files": [], "requested_networks": null}, "_context_auth_token": "deb477655fba448e85199f7e559da77a", "_context_is_admin": false, "_context_project_id": "df3827f76f714b1e8f31675caf84ae9d", "_context_timestamp": "2012-03-24T01:39:52.089383", "_context_user_id": "abe21eb7e6884547810f0a43c216e6a6", "method": "run_instance", "_context_remote_address": "192.168.1.41"}'

[x] Received '{"_context_roles": ["Member", "admin"], "_context_request_id": "req-45e6c2af-52c7-4de3-af6c-6b2f7520cfd5", "_context_read_deleted": "no", "args": {"instance_uuid": "40b5a1c5-bd4f-40ee-ae0a-73e0bc927431", "requested_networks": null, "is_first_time": true, "admin_password": "SKohh79r956J", "injected_files": []}, "_context_auth_token": "deb477655fba448e85199f7e559da77a", "_context_is_admin": true, "_context_project_id": "df3827f76f714b1e8f31675caf84ae9d", "_context_timestamp": "2012-03-24T01:39:52.089383", "_context_user_id": "abe21eb7e6884547810f0a43c216e6a6", "method": "run_instance", "_context_remote_address": "192.168.1.41"}'

[x] Received '{"_context_roles": ["Member", "admin"], "_msg_id": "f40e21507437492f97a02cd25415498a", "_context_read_deleted": "no", "_context_request_id": "req-45e6c2af-52c7-4de3-af6c-6b2f7520cfd5", "args": {"instance_uuid": "40b5a1c5-bd4f-40ee-ae0a-73e0bc927431", "vpn": false, "requested_networks": null, "instance_id": 7, "host": "ubuntu", "rxtx_factor": 1.0, "project_id": "df3827f76f714b1e8f31675caf84ae9d"}, "_context_auth_token": "deb477655fba448e85199f7e559da77a", "_context_is_admin": true, "_context_project_id": "df3827f76f714b1e8f31675caf84ae9d", "_context_timestamp": "2012-03-24T01:39:52.089383", "_context_user_id": "abe21eb7e6884547810f0a43c216e6a6", "method": "allocate_for_instance", "_context_remote_address": "192.168.1.41"}'

[x] Received '{"_context_roles": ["admin"], "_msg_id": "96c3d16edf894a89ae85ed90b0a0858b", "_context_read_deleted": "no", "_context_request_id": "req-81c9353b-f912-408e-a297-0e8ad6b8fe10", "args": {"instance_id": 6, "instance_uuid": "e3ad17e6-dd59-4b67-a7d0-e3812f96c2d7", "host": "ubuntu", "project_id": "c290118b14564257be26a2cb901721a2", "rxtx_factor": 1.0}, "_context_auth_token": null, "_context_is_admin": true, "_context_project_id": null, "_context_timestamp": "2012-03-24T01:40:01.390757", "_context_user_id": null, "method": "get_instance_nw_info", "_context_remote_address": null}'

[x] Received '{"_context_roles": ["admin"], "_context_request_id": "req-d0707421-7f4e-4f1f-bf89-109ca4625ca5", "_context_read_deleted": "no", "args": {"address": "10.0.0.2"}, "_context_auth_token": null, "_context_is_admin": true, "_context_project_id": null, "_context_timestamp": "2012-03-24T01:40:53.338021", "_context_user_id": null, "method": "lease_fixed_ip", "_context_remote_address": null}'

[x] Received '{"_context_roles": ["admin"], "_msg_id": "38ad50d1abf445118c60017ee03851f6", "_context_read_deleted": "no", "_context_request_id": "req-51cd0d75-17e5-414b-affd-1ca2060cc8cb", "args": {"instance_id": 7, "instance_uuid": "40b5a1c5-bd4f-40ee-ae0a-73e0bc927431", "host": "ubuntu", "project_id": "df3827f76f714b1e8f31675caf84ae9d", "rxtx_factor": 1.0}, "_context_auth_token": null, "_context_is_admin": true, "_context_project_id": null, "_context_timestamp": "2012-03-24T01:41:07.580157", "_context_user_id": null, "method": "get_instance_nw_info", "_context_remote_address": null}'

On Mon, Apr 23, 2012 at 9:23 PM, Doug Hellmann <doug.hellmann@dreamhost.com> wrote:


On Mon, Apr 23, 2012 at 1:50 PM, Brian Schott <brian.schott@nimbisservices.com> wrote:
So, we could build on this. No reason to reinvent, but we might want to expand the number of events. I'm concerned about things like what happens when flavors change over time. Maybe the answer is, always append to the flavor/instance-type table. The code I remember and the admin interface that Ken wrote allowed you to modify flavors. That would break billing unless you also track flavor modifications.

That seems like a situation where you would want to denormalize the billing database and record the flavor details along with the rest of the creation event data.

Doug

StackTach is a Django-based web interface for capturing, displaying and
navigating OpenStack notifications

https://github.com/rackspace/stacktach

-S


On 04/23/2012 04:26 PM, Luis Gervaso wrote:
> Joshua,
>
> I have performed a create instance operation and here is an example data
> obtained from stable/essex rabbitmq nova catch all exchange.
>
> [*] Waiting for messages. To exit press CTRL+C
>
> [x] Received '{"_context_roles": ["admin"], "_msg_id":
> "a2d13735baad4613b89c6132e0fa8302", "_context_read_deleted": "no",
> "_context_request_id": "req-d7ffbe78-7a9c-4d20-9ac5-3e56951526fe",
> "args": {"instance_id": 6, "instance_uuid":
> "e3ad17e6-dd59-4b67-a7d0-e3812f96c2d7", "host": "ubuntu", "project_id":
> "c290118b14564257be26a2cb901721a2", "rxtx_factor": 1.0},
> "_context_auth_token": null, "_context_is_admin": true,
> "_context_project_id": null, "_context_timestamp":
> "2012-03-24T01:36:48.774891", "_context_user_id": null, "method":
> "get_instance_nw_info", "_context_remote_address": null}'
>
> [x] Received '{"_context_roles": ["admin"], "_msg_id":
> "a1cb1cf61e5441c2a772b29d3cd54202", "_context_read_deleted": "no",
> "_context_request_id": "req-db34ba32-8bd9-4cd5-b7b5-43705a9e258e",
> "args": {"instance_id": 6, "instance_uuid":
> "e3ad17e6-dd59-4b67-a7d0-e3812f96c2d7", "host": "ubuntu", "project_id":
> "c290118b14564257be26a2cb901721a2", "rxtx_factor": 1.0},
> "_context_auth_token": null, "_context_is_admin": true,
> "_context_project_id": null, "_context_timestamp":
> "2012-03-24T01:37:50.463586", "_context_user_id": null, "method":
> "get_instance_nw_info", "_context_remote_address": null}'
>
> [x] Received '{"_context_roles": ["admin"], "_msg_id":
> "ebb0b1c340de4024a22eafec9d0a2d66", "_context_read_deleted": "no",
> "_context_request_id": "req-ddb51b2b-a29f-4aad-909d-3f7f79f053c4",
> "args": {"instance_id": 6, "instance_uuid":
> "e3ad17e6-dd59-4b67-a7d0-e3812f96c2d7", "host": "ubuntu", "project_id":
> "c290118b14564257be26a2cb901721a2", "rxtx_factor": 1.0},
> "_context_auth_token": null, "_context_is_admin": true,
> "_context_project_id": null, "_context_timestamp":
> "2012-03-24T01:38:59.217333", "_context_user_id": null, "method":
> "get_instance_nw_info", "_context_remote_address": null}'
>
> [x] Received '{"_context_roles": ["Member"], "_msg_id":
> "729535c00d224414a98286e9ce3475a9", "_context_read_deleted": "no",
> "_context_request_id": "req-b056a8cc-3542-41a9-9e58-8fb592086264",
> "_context_auth_token": "deb477655fba448e85199f7e559da77a",
> "_context_is_admin": false, "_context_project_id":
> "df3827f76f714b1e8f31675caf84ae9d", "_context_timestamp":
> "2012-03-24T01:39:19.813393", "_context_user_id":
> "abe21eb7e6884547810f0a43c216e6a6", "method":
> "get_floating_ips_by_project", "_context_remote_address": "192.168.1.41"}'
>
> [x] Received '{"_context_roles": ["Member", "admin"],
> "_context_request_id": "req-45e6c2af-52c7-4de3-af6c-6b2f7520cfd5",
> "_context_read_deleted": "no", "args": {"request_spec":
> {"num_instances": 1, "block_device_mapping": [], "image": {"status":
> "active", "name": "cirros-0.3.0-x86_64-uec", "deleted": false,
> "container_format": "ami", "created_at": "2012-03-20 17:37:08",
> "disk_format": "ami", "updated_at": "2012-03-20 17:37:08", "properties":
> {"kernel_id": "6b700d25-3293-420a-82e4-8247d4b0da2a", "ramdisk_id":
> "22b10c35-c868-4470-84ef-54ae9f17a977"}, "min_ram": "0", "checksum":
> "2f81976cae15c16ef0010c51e3a6c163", "min_disk": "0", "is_public": true,
> "deleted_at": null, "id": "f7d4bea2-2aed-4bf3-a5cb-db6a34c4a525",
> "size": 25165824}, "instance_type": {"root_gb": 0, "name": "m1.tiny",
> "deleted": false, "created_at": null, "ephemeral_gb": 0, "updated_at":
> null, "memory_mb": 512, "vcpus": 1, "flavorid": "1", "swap": 0,
> "rxtx_factor": 1.0, "extra_specs": {}, "deleted_at": null,
> "vcpu_weight": null, "id": 2}, "instance_properties": {"vm_state":
> "building", "ephemeral_gb": 0, "access_ip_v6": null, "access_ip_v4":
> null, "kernel_id": "6b700d25-3293-420a-82e4-8247d4b0da2a", "key_name":
> "testssh", "ramdisk_id": "22b10c35-c868-4470-84ef-54ae9f17a977",
> "instance_type_id": 2, "user_data": "dGhpcyBpcyBteSB1c2VyIGRhdGE=",
> "vm_mode": null, "display_name": "eureka", "config_drive_id": "",
> "reservation_id": "r-xtzjx50j", "key_data": "ssh-rsa
> AAAAB3NzaC1yc2EAAAADAQABAAAAgQDJ31tdayh1xnAY+JO/ZVdg5L83CsIU7qaOmFubdH7zlg2jjS9JmkPNANj99zx+UHg5F5JKGMef9M8VP/V89D5g0oIjIJtBdFpKOScBo3yJ1vteW5ItImH8h9TldymHf+CWNVY1oNNqzXqAb41xwUUDNvgeXHRZNnE6tmwZO0oC1Q==
> stack@ubuntu\n", "root_gb": 0, "user_id":
> "abe21eb7e6884547810f0a43c216e6a6", "uuid":
> "40b5a1c5-bd4f-40ee-ae0a-73e0bc927431", "root_device_name": null,
> "availability_zone": null, "launch_time": "2012-03-24T01:39:52Z",
> "metadata": {}, "display_description": "eureka", "memory_mb": 512,
> "launch_index": 0, "vcpus": 1, "locked": false, "image_ref":
> "f7d4bea2-2aed-4bf3-a5cb-db6a34c4a525", "architecture": null,
> "power_state": 0, "auto_disk_config": null, "progress": 0, "os_type":
> null, "project_id": "df3827f76f714b1e8f31675caf84ae9d", "config_drive":
> ""}, "security_group": ["default"]}, "is_first_time": true,
> "filter_properties": {"scheduler_hints": {}}, "topic": "compute",
> "admin_password": "SKohh79r956J", "injected_files": [],
> "requested_networks": null}, "_context_auth_token":
> "deb477655fba448e85199f7e559da77a", "_context_is_admin": false,
> "_context_project_id": "df3827f76f714b1e8f31675caf84ae9d",
> "_context_timestamp": "2012-03-24T01:39:52.089383", "_context_user_id":
> "abe21eb7e6884547810f0a43c216e6a6", "method": "run_instance",
> "_context_remote_address": "192.168.1.41"}'
>
> [x] Received '{"_context_roles": ["Member", "admin"],
> "_context_request_id": "req-45e6c2af-52c7-4de3-af6c-6b2f7520cfd5",
> "_context_read_deleted": "no", "args": {"instance_uuid":
> "40b5a1c5-bd4f-40ee-ae0a-73e0bc927431", "requested_networks": null,
> "is_first_time": true, "admin_password": "SKohh79r956J",
> "injected_files": []}, "_context_auth_token":
> "deb477655fba448e85199f7e559da77a", "_context_is_admin": true,
> "_context_project_id": "df3827f76f714b1e8f31675caf84ae9d",
> "_context_timestamp": "2012-03-24T01:39:52.089383", "_context_user_id":
> "abe21eb7e6884547810f0a43c216e6a6", "method": "run_instance",
> "_context_remote_address": "192.168.1.41"}'
>
> [x] Received '{"_context_roles": ["Member", "admin"], "_msg_id":
> "f40e21507437492f97a02cd25415498a", "_context_read_deleted": "no",
> "_context_request_id": "req-45e6c2af-52c7-4de3-af6c-6b2f7520cfd5",
> "args": {"instance_uuid": "40b5a1c5-bd4f-40ee-ae0a-73e0bc927431", "vpn":
> false, "requested_networks": null, "instance_id": 7, "host": "ubuntu",
> "rxtx_factor": 1.0, "project_id": "df3827f76f714b1e8f31675caf84ae9d"},
> "_context_auth_token": "deb477655fba448e85199f7e559da77a",
> "_context_is_admin": true, "_context_project_id":
> "df3827f76f714b1e8f31675caf84ae9d", "_context_timestamp":
> "2012-03-24T01:39:52.089383", "_context_user_id":
> "abe21eb7e6884547810f0a43c216e6a6", "method": "allocate_for_instance",
> "_context_remote_address": "192.168.1.41"}'
>
> [x] Received '{"_context_roles": ["admin"], "_msg_id":
> "96c3d16edf894a89ae85ed90b0a0858b", "_context_read_deleted": "no",
> "_context_request_id": "req-81c9353b-f912-408e-a297-0e8ad6b8fe10",
> "args": {"instance_id": 6, "instance_uuid":
> "e3ad17e6-dd59-4b67-a7d0-e3812f96c2d7", "host": "ubuntu", "project_id":
> "c290118b14564257be26a2cb901721a2", "rxtx_factor": 1.0},
> "_context_auth_token": null, "_context_is_admin": true,
> "_context_project_id": null, "_context_timestamp":
> "2012-03-24T01:40:01.390757", "_context_user_id": null, "method":
> "get_instance_nw_info", "_context_remote_address": null}'
>
> [x] Received '{"_context_roles": ["admin"], "_context_request_id":
> "req-d0707421-7f4e-4f1f-bf89-109ca4625ca5", "_context_read_deleted":
> "no", "args": {"address": "10.0.0.2"}, "_context_auth_token": null,
> "_context_is_admin": true, "_context_project_id": null,
> "_context_timestamp": "2012-03-24T01:40:53.338021", "_context_user_id":
> null, "method": "lease_fixed_ip", "_context_remote_address": null}'
>
> [x] Received '{"_context_roles": ["admin"], "_msg_id":
> "38ad50d1abf445118c60017ee03851f6", "_context_read_deleted": "no",
> "_context_request_id": "req-51cd0d75-17e5-414b-affd-1ca2060cc8cb",
> "args": {"instance_id": 7, "instance_uuid":
> "40b5a1c5-bd4f-40ee-ae0a-73e0bc927431", "host": "ubuntu", "project_id":
> "df3827f76f714b1e8f31675caf84ae9d", "rxtx_factor": 1.0},
> "_context_auth_token": null, "_context_is_admin": true,
> "_context_project_id": null, "_context_timestamp":
> "2012-03-24T01:41:07.580157", "_context_user_id": null, "method":
> "get_instance_nw_info", "_context_remote_address": null}'
>
>
> On Mon, Apr 23, 2012 at 9:23 PM, Doug Hellmann
> <doug.hellmann@dreamhost.com <mailto:doug.hellmann@dreamhost.com>> wrote:
>
>
>
> On Mon, Apr 23, 2012 at 1:50 PM, Brian Schott
> <brian.schott@nimbisservices.com
> <mailto:brian.schott@nimbisservices.com>> wrote:
>
> So, we could build on this. No reason to reinvent, but we might
> want to expand the number of events. I'm concerned about things
> like what happens when flavors change over time. Maybe the
> answer is, always append to the flavor/instance-type table. The
> code I remember and the admin interface that Ken wrote allowed
> you to modify flavors. That would break billing unless you also
> track flavor modifications.
>
>
> That seems like a situation where you would want to denormalize the
> billing database and record the flavor details along with the rest
> of the creation event data.
>
> Doug
>
>
>
>
> --
> -------------------------------------------
> Luis Alberto Gervaso Martin
> Woorea Solutions, S.L
> CEO & CTO
> mobile: (+34) 627983344
> luis@ <mailto:luis.gervaso@gmail.com>woorea.es <http://woorea.es/>
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

This looks like just the standard RPC traffic.
You need to turn notifications on
(set:
notification_driver=nova.notifier.rabbit_notifier
in nova's config file)

and listen on the notification.* queues



On Apr 23, 2012, at 2:26 PM, Luis Gervaso wrote:

Joshua,

I have performed a create instance operation and here is an example data obtained from stable/essex rabbitmq nova catch all exchange.

[*] Waiting for messages. To exit press CTRL+C

[x] Received '{"_context_roles": ["admin"], "_msg_id": "a2d13735baad4613b89c6132e0fa8302", "_context_read_deleted": "no", "_context_request_id": "req-d7ffbe78-7a9c-4d20-9ac5-3e56951526fe", "args": {"instance_id": 6, "instance_uuid": "e3ad17e6-dd59-4b67-a7d0-e3812f96c2d7", "host": "ubuntu", "project_id": "c290118b14564257be26a2cb901721a2", "rxtx_factor": 1.0}, "_context_auth_token": null, "_context_is_admin": true, "_context_project_id": null, "_context_timestamp": "2012-03-24T01:36:48.774891", "_context_user_id": null, "method": "get_instance_nw_info", "_context_remote_address": null}'

[x] Received '{"_context_roles": ["admin"], "_msg_id": "a1cb1cf61e5441c2a772b29d3cd54202", "_context_read_deleted": "no", "_context_request_id": "req-db34ba32-8bd9-4cd5-b7b5-43705a9e258e", "args": {"instance_id": 6, "instance_uuid": "e3ad17e6-dd59-4b67-a7d0-e3812f96c2d7", "host": "ubuntu", "project_id": "c290118b14564257be26a2cb901721a2", "rxtx_factor": 1.0}, "_context_auth_token": null, "_context_is_admin": true, "_context_project_id": null, "_context_timestamp": "2012-03-24T01:37:50.463586", "_context_user_id": null, "method": "get_instance_nw_info", "_context_remote_address": null}'

[x] Received '{"_context_roles": ["admin"], "_msg_id": "ebb0b1c340de4024a22eafec9d0a2d66", "_context_read_deleted": "no", "_context_request_id": "req-ddb51b2b-a29f-4aad-909d-3f7f79f053c4", "args": {"instance_id": 6, "instance_uuid": "e3ad17e6-dd59-4b67-a7d0-e3812f96c2d7", "host": "ubuntu", "project_id": "c290118b14564257be26a2cb901721a2", "rxtx_factor": 1.0}, "_context_auth_token": null, "_context_is_admin": true, "_context_project_id": null, "_context_timestamp": "2012-03-24T01:38:59.217333", "_context_user_id": null, "method": "get_instance_nw_info", "_context_remote_address": null}'

[x] Received '{"_context_roles": ["Member"], "_msg_id": "729535c00d224414a98286e9ce3475a9", "_context_read_deleted": "no", "_context_request_id": "req-b056a8cc-3542-41a9-9e58-8fb592086264", "_context_auth_token": "deb477655fba448e85199f7e559da77a", "_context_is_admin": false, "_context_project_id": "df3827f76f714b1e8f31675caf84ae9d", "_context_timestamp": "2012-03-24T01:39:19.813393", "_context_user_id": "abe21eb7e6884547810f0a43c216e6a6", "method": "get_floating_ips_by_project", "_context_remote_address": "192.168.1.41"}'

[x] Received '{"_context_roles": ["Member", "admin"], "_context_request_id": "req-45e6c2af-52c7-4de3-af6c-6b2f7520cfd5", "_context_read_deleted": "no", "args": {"request_spec": {"num_instances": 1, "block_device_mapping": [], "image": {"status": "active", "name": "cirros-0.3.0-x86_64-uec", "deleted": false, "container_format": "ami", "created_at": "2012-03-20 17:37:08", "disk_format": "ami", "updated_at": "2012-03-20 17:37:08", "properties": {"kernel_id": "6b700d25-3293-420a-82e4-8247d4b0da2a", "ramdisk_id": "22b10c35-c868-4470-84ef-54ae9f17a977"}, "min_ram": "0", "checksum": "2f81976cae15c16ef0010c51e3a6c163", "min_disk": "0", "is_public": true, "deleted_at": null, "id": "f7d4bea2-2aed-4bf3-a5cb-db6a34c4a525", "size": 25165824}, "instance_type": {"root_gb": 0, "name": "m1.tiny", "deleted": false, "created_at": null, "ephemeral_gb": 0, "updated_at": null, "memory_mb": 512, "vcpus": 1, "flavorid": "1", "swap": 0, "rxtx_factor": 1.0, "extra_specs": {}, "deleted_at": null, "vcpu_weight": null, "id": 2}, "instance_properties": {"vm_state": "building", "ephemeral_gb": 0, "access_ip_v6": null, "access_ip_v4": null, "kernel_id": "6b700d25-3293-420a-82e4-8247d4b0da2a", "key_name": "testssh", "ramdisk_id": "22b10c35-c868-4470-84ef-54ae9f17a977", "instance_type_id": 2, "user_data": "dGhpcyBpcyBteSB1c2VyIGRhdGE=", "vm_mode": null, "display_name": "eureka", "config_drive_id": "", "reservation_id": "r-xtzjx50j", "key_data": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAgQDJ31tdayh1xnAY+JO/ZVdg5L83CsIU7qaOmFubdH7zlg2jjS9JmkPNANj99zx+UHg5F5JKGMef9M8VP/V89D5g0oIjIJtBdFpKOScBo3yJ1vteW5ItImH8h9TldymHf+CWNVY1oNNqzXqAb41xwUUDNvgeXHRZNnE6tmwZO0oC1Q== stack@ubuntu\n", "root_gb": 0, "user_id": "abe21eb7e6884547810f0a43c216e6a6", "uuid": "40b5a1c5-bd4f-40ee-ae0a-73e0bc927431", "root_device_name": null, "availability_zone": null, "launch_time": "2012-03-24T01:39:52Z", "metadata": {}, "display_description": "eureka", "memory_mb": 512, "launch_index": 0, "vcpus": 1, "locked": false, "image_ref": "f7d4bea2-2aed-4bf3-a5cb-db6a34c4a525", "architecture": null, "power_state": 0, "auto_disk_config": null, "progress": 0, "os_type": null, "project_id": "df3827f76f714b1e8f31675caf84ae9d", "config_drive": ""}, "security_group": ["default"]}, "is_first_time": true, "filter_properties": {"scheduler_hints": {}}, "topic": "compute", "admin_password": "SKohh79r956J", "injected_files": [], "requested_networks": null}, "_context_auth_token": "deb477655fba448e85199f7e559da77a", "_context_is_admin": false, "_context_project_id": "df3827f76f714b1e8f31675caf84ae9d", "_context_timestamp": "2012-03-24T01:39:52.089383", "_context_user_id": "abe21eb7e6884547810f0a43c216e6a6", "method": "run_instance", "_context_remote_address": "192.168.1.41"}'

[x] Received '{"_context_roles": ["Member", "admin"], "_context_request_id": "req-45e6c2af-52c7-4de3-af6c-6b2f7520cfd5", "_context_read_deleted": "no", "args": {"instance_uuid": "40b5a1c5-bd4f-40ee-ae0a-73e0bc927431", "requested_networks": null, "is_first_time": true, "admin_password": "SKohh79r956J", "injected_files": []}, "_context_auth_token": "deb477655fba448e85199f7e559da77a", "_context_is_admin": true, "_context_project_id": "df3827f76f714b1e8f31675caf84ae9d", "_context_timestamp": "2012-03-24T01:39:52.089383", "_context_user_id": "abe21eb7e6884547810f0a43c216e6a6", "method": "run_instance", "_context_remote_address": "192.168.1.41"}'

[x] Received '{"_context_roles": ["Member", "admin"], "_msg_id": "f40e21507437492f97a02cd25415498a", "_context_read_deleted": "no", "_context_request_id": "req-45e6c2af-52c7-4de3-af6c-6b2f7520cfd5", "args": {"instance_uuid": "40b5a1c5-bd4f-40ee-ae0a-73e0bc927431", "vpn": false, "requested_networks": null, "instance_id": 7, "host": "ubuntu", "rxtx_factor": 1.0, "project_id": "df3827f76f714b1e8f31675caf84ae9d"}, "_context_auth_token": "deb477655fba448e85199f7e559da77a", "_context_is_admin": true, "_context_project_id": "df3827f76f714b1e8f31675caf84ae9d", "_context_timestamp": "2012-03-24T01:39:52.089383", "_context_user_id": "abe21eb7e6884547810f0a43c216e6a6", "method": "allocate_for_instance", "_context_remote_address": "192.168.1.41"}'

[x] Received '{"_context_roles": ["admin"], "_msg_id": "96c3d16edf894a89ae85ed90b0a0858b", "_context_read_deleted": "no", "_context_request_id": "req-81c9353b-f912-408e-a297-0e8ad6b8fe10", "args": {"instance_id": 6, "instance_uuid": "e3ad17e6-dd59-4b67-a7d0-e3812f96c2d7", "host": "ubuntu", "project_id": "c290118b14564257be26a2cb901721a2", "rxtx_factor": 1.0}, "_context_auth_token": null, "_context_is_admin": true, "_context_project_id": null, "_context_timestamp": "2012-03-24T01:40:01.390757", "_context_user_id": null, "method": "get_instance_nw_info", "_context_remote_address": null}'

[x] Received '{"_context_roles": ["admin"], "_context_request_id": "req-d0707421-7f4e-4f1f-bf89-109ca4625ca5", "_context_read_deleted": "no", "args": {"address": "10.0.0.2"}, "_context_auth_token": null, "_context_is_admin": true, "_context_project_id": null, "_context_timestamp": "2012-03-24T01:40:53.338021", "_context_user_id": null, "method": "lease_fixed_ip", "_context_remote_address": null}'

[x] Received '{"_context_roles": ["admin"], "_msg_id": "38ad50d1abf445118c60017ee03851f6", "_context_read_deleted": "no", "_context_request_id": "req-51cd0d75-17e5-414b-affd-1ca2060cc8cb", "args": {"instance_id": 7, "instance_uuid": "40b5a1c5-bd4f-40ee-ae0a-73e0bc927431", "host": "ubuntu", "project_id": "df3827f76f714b1e8f31675caf84ae9d", "rxtx_factor": 1.0}, "_context_auth_token": null, "_context_is_admin": true, "_context_project_id": null, "_context_timestamp": "2012-03-24T01:41:07.580157", "_context_user_id": null, "method": "get_instance_nw_info", "_context_remote_address": null}'

On Mon, Apr 23, 2012 at 9:23 PM, Doug Hellmann <doug.hellmann@dreamhost.com<mailto:doug.hellmann@dreamhost.com>> wrote:


On Mon, Apr 23, 2012 at 1:50 PM, Brian Schott <brian.schott@nimbisservices.com<mailto:brian.schott@nimbisservices.com>> wrote:
So, we could build on this. No reason to reinvent, but we might want to expand the number of events. I'm concerned about things like what happens when flavors change over time. Maybe the answer is, always append to the flavor/instance-type table. The code I remember and the admin interface that Ken wrote allowed you to modify flavors. That would break billing unless you also track flavor modifications.

That seems like a situation where you would want to denormalize the billing database and record the flavor details along with the rest of the creation event data.

Doug




--
-------------------------------------------
Luis Alberto Gervaso Martin
Woorea Solutions, S.L
CEO & CTO
mobile: (+34) 627983344
luis@<mailto:luis.gervaso@gmail.com>woorea.es<http://woorea.es/>

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net<mailto:openstack@lists.launchpad.net>
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

--
Monsyne M. Dragon
OpenStack/Nova
cell 210-441-0965
work x 5014190