[00:17:46] <trentster> killfill: are your chunter nodes still logging to debug.log ?
[00:23:03] *** fxhp has joined #project-FiFo
[00:23:18] <Harvi> hi trenster i think i did it only seems to be an issue when i am downloading more than three datasets to the local repository after about an hour the system is back to normal
[00:23:34] <Harvi> Im from London UK killfill
[00:24:37] <Harvi> Licenser may i congratulate you on a fantastic piece of software im very impressed i also like the elegant use of angular js so hats off to the person who did the gui
[00:24:59] <Harvi> Do you guys have plans for implementing a billing api for the platform?
[00:30:59] <Harvi> I was thinking that http://www.boxbilling.com would be a great open source billing complement for the platform in order to use projectfifo in a multi tenanted commercial environment
[01:01:34] <Harvi> a quick question how important is including the "zonename" is the setupfifo.json as it doesnt seem to be in the installation guidelines?
[01:09:14] <trentster> Harvi: killfill did the UI, and it is awesome!
[01:09:50] <trentster> Harvi: um no billing api integration yet afaik, altho it would be great to have.
[01:11:49] <trentster> Harvi: yeah pulling in datasets, takes time. each dataset is chunk into 3 redundant chunks. Which takes time. If you only run a single fifo zone all 3 chunks gets put into the same riak store.
[01:12:05] <trentster> which consumes about 3X space per dataset.
[01:12:31] <trentster> Best is to import a couple at a tiem and wait for them to finish, with a "tick" next to them in the list
[01:12:39] *** jim80net has quit IRC
[01:14:24] *** jlove has joined #project-FiFo
[01:14:54] <Harvi> Thanks for the heads up Trenster I havent got a stable riak enviroment yet and am just testing my cloud locally on 4 3770 based intel systems with 32gb ram each. I will be moving it to a Supermicro Based Superblade solution with a in house built zfs/glusterfs san solution
[01:17:06] <trentster> Harvi: sounds decent! you got a particular custom itch you are trying to scratch with the glusterfs stuff?
[01:19:16] *** jlove has quit IRC
[01:21:58] <Harvi> yes my business partner set me a challenge to build a scale out storage platform that was more reliable and cost effective and not hooked up to prism et al. So based on a custom hardware design that borrows a lot on backblaze open storage pod concept we have built a petabyte scale storage solution that uses smartos/omnios to build a storage platform its early days at the moment but it looks very promising our cost per gigabyte is
[01:22:49] <Harvi> so basically an angularjs web based secure cloud storage solution and a python client for mac/windows
[01:25:08] <Harvi> how do i go about rolling a dataset to import into project fifo?
[01:25:12] <trentster> wow, sounds amazing!
[01:25:25] <trentster> zone or kvm dataset?
[01:27:08] <Harvi> testing both at the moment im currenly testing using an array of 200 usb sticks connected to the machines 50 a machine for proof of concept and will be building the storage pods and do a full performance and reliability analysis over the coming couple of months
[01:27:25] <Harvi> I will be trying both zone and kvm
[01:28:24] <Harvi> zones seem to have an inherent performance benefit with regards to network IO and based on our design that seems to be the bottle neck and kvm based solutions cant seem to come anywhere close the the performance benefits of using a zone
[01:28:27] <trentster> The backblaze stuff was really something back when they had open design for the chassis. Are you busy writing your own angular UI for the storage stuff?
[01:29:28] <trentster> Harvi: basic rule is stay away from KVM at all costs unless you specifically need it. For your business model/use case does not sound like you need kvm.
[01:29:48] <trentster> To create a zone dataset is pretty simple.
[01:29:57] <trentster> 1- create a zone vm
[01:30:00] <Harvi> The design is still open and on version three although i have modded it using solidworks to work better for our use case replacing the sata backplanes with sas and virtually no cost difference and removing the backplane bottle neck
[01:30:15] <trentster> 2- install your apps etc and get it just right
[01:30:42] <trentster> 3- from within the zone, run sm-prepare-image which gets it ready to be created for dataset use.
[01:30:48] <trentster> 4 - shut down zone
[01:31:14] <trentster> 5 - within fifo create snapshot e.g uuid@sm-prepared
[01:31:55] <trentster> 6 - within fifo click on imaging section and create dataset from snapshot. It will then pull the dataset into fifo dataset server and you can then deploy
[01:32:19] <Harvi> trentster i cant thank you enough for sharing this information with me im looking forward to rolling my own datasets
[01:32:22] <trentster> tip: you can always get your datasets out of fifo dataset server and archive them etc, using fifoadm datasets export
[01:32:32] <Harvi> nice one
[01:32:57] <trentster> np, glad to be of some help ;-)
[01:34:17] <trentster> sas backplanes can be tricky, I got a 16 bay supermicro server at home, and half the backplane is kinda unstable, with drives falling out of zpool when under load.
[01:34:33] <trentster> The other half of the backplane is rock solid. very weird indeed.
[01:38:05] <Harvi> ahh yes i have heard about problems will share my findings with you once the hard gets shipped over basically the design im using accomodates 64 drives using three sas backplanes in 4u
[01:38:37] <Harvi> *Hardware
[01:40:58] <trentster> so you not using jbods, each storage node is gonna be a compute node?
[01:42:46] <Harvi> yes each node is going to be 192TB Raw storage with its own compute power, the reason for that is the each node will have 1TB of Memory and Dual SSDs for L2ARC and ZIL
[01:43:12] <Harvi> Thats to support my compressio/encryption/dedupe targets for each node
[01:43:24] <trentster> you know, you are making me jealous ;-)
[01:43:45] <Harvi> very cheap to build
[01:43:55] <trentster> yeah right
[01:44:04] *** while1eq1_ has joined #project-FiFo
[01:44:19] <arekinath> compared to buying oracle, it's peanuts. ;)
[01:44:25] <trentster> I guess cheap is relative, all comes down to the revenue model
[01:44:46] <Harvi> I have setup a side business to sell the boxes to bioinformatics/big data/external firms which allows me to basically make a profit and get my own hardware free
[01:45:01] <trentster> jumping off a building is less painful than going the oracle route :-P
[01:45:28] <arekinath> trentster: yeah, but in most companies the people who make the oracle decision are not the ones who have to deal with the pain. :)
[01:46:43] *** while1eq1_ has quit IRC
[01:47:04] <Harvi> currently for my angularui im creating a perl based scraper which then gets wrapped into a Mongo/Express/Node/Angular App to present the front end
[01:47:22] <trentster> yeah, and you have to charge your own customers larger than required $$ to make the business offering viable to pay for oracle hardware/ software / support and greed costs
[01:47:43] <Harvi> absolutley
[01:49:01] *** while1eq1 has quit IRC
[01:49:39] *** while1eq1 has joined #project-FiFo
[01:51:13] <trentster> Harvi: sounds like you are going to cojur into existence some interesting stuff!, hopefully some of that may be relevant to improving some of fifo's feature set.
[01:51:22] <trentster> *conjur
[01:52:17] <Harvi> I hope so it has me very excited and i would love to help contribute and help build the fifo orchestration suite right now im just getting my teeth into it!
[01:54:26] <trentster> Yeah fifo is amazing!, it has a couple of areas with some rough edges, and the more people using / contributing the better it will be. :^)
[01:55:14] <Harvi> im also doing quite a lot of work with chef solo which fits very nicely with the way that smartos hypervisors work can see some positive synergies into automating chef solo into the hypervisor build
[01:56:23] <trentster> Yeah, a lot of cool stuff may also come along when fifo incorporates its own PXE boot stuff and can provision new nodes and auto integrate them.
[01:56:49] <trentster> Harvi: also check out the workflow rules in fifo, they are quite powerful.
[01:57:15] <trentster> you can designate tags/attributes to nodes and then get the auto provisioning algorithm to make decisions based on them
[01:57:20] <killfill> sounds grate.. Harvi and your playing with fifo to see if you can use of for building uppon it some kind of platform?
[01:57:33] <Harvi> I have been working with PXE and currently i autodeploy all my hypervisors on boot using PXE and Bootp just giving my pxe boot server the mac addresses
[01:57:57] <trentster> e.g if the dataset is branded for customer X who is not allowed to have his vm on shared hardware deploy only to nodes branded as customer X nodes.
[01:58:16] <trentster> or only provision this package type on this hardware node type.
[01:58:21] <trentster> blah blah.
[01:58:39] <Harvi> Yes im building a private cloud with its own storage backend to launch a suite of IaaS, PaaS, SaaS businesses and websites i moved over to it from working with openstack and eucalyptus and am loving fifo+smartos
[01:59:42] <trentster> yeah coming from eucalyptus/openstack there is a lot to love with SmartOS + FiFO
[02:00:33] <killfill> what didnt you like about thouse?
[02:00:55] <trentster> The performance you get with zones and the introspection + analytics is a deal maker ;-)
[02:01:32] <trentster> killfill: too many moving parts, too many dedicated server roles, and centralised storage.
[02:01:57] <trentster> I think I would rather use Cloudstack vs openstack if I was forced to make the decision.
[02:02:10] <Harvi> openstack is extemely powerful but you need a team of people to really get the best out of that platform and needs a lot of customisation and extra coding too much complexity for mid sized projects and unless your familiar with cloud design patterns you can make some expensive mistakes time and money wise
[02:02:51] <trentster> plus everyone forgets about the stability of solaris/illumos vs Linux - but "I dont want to get a OS war started here"
[02:02:57] <Harvi> its like openstack in a box
[02:03:30] <Harvi> indeed im platform agnostic and use the best tools to hand
[02:04:02] <trentster> yeah me too, but more and more I land up reaching for the same tools most of the time.
[02:04:55] <trentster> Given a choice I prefer illumos / FreeBSD and if not then Linux
[02:05:05] <Harvi> from openstack i moved to proxmox for this particular usecase and accidently discovered fifo and smartos and i was converted hehe
[02:05:36] <trentster> Harvi: heh yeah I also came from proxmox to smartos.
[02:06:27] <trentster> I had been waiting for a native ZFS vm/cloud solution, and always wished I could use my nappit storage nodes for kvm virtualization.
[02:06:41] <trentster> when smartos came along I had a big grin on my face ;-)
[02:07:32] <Harvi> hehe likewise
[02:09:07] <Harvi> have you guys had much luck running OmniOS in Zones?
[02:09:21] <Harvi> on a SmartOS hypervisor?
[02:09:23] <trentster> on SmartOS?
[02:09:41] <trentster> erm no, haven't tried.
[02:10:11] <Harvi> that would really open up the nappit route as nappit runs natively on OmniOS
[02:10:23] <trentster> but there is talk about possibly adding omnios support to fifo, plus a chunter agent for Omnios, so that you can deploy zones on omnios nodes with fifo
[02:10:47] <Harvi> that would be perfect
[02:12:36] <trentster> Harvi: yeah, it would be cool, but nappit expects root access to zpool and disks etc, its tricky, that being said I ahve gotten amazing performance by using local zones with lofs mounts in them, especially for mac backups, the afp speed is blazingly fast, saturates a my Gig link. I wrote a blog post about it awhile back if you are interested
[02:13:18] <Harvi> yes do you have the link?
[02:15:18] <Harvi> ahh thats your blog hehe i think i read that a couple of weeks back yes very impressive
[02:15:38] <trentster> oh, you have read the article previously?
[02:16:18] <Harvi> yes i have a printout of it in a folder on my desk along with the rest of my zfs/smartos findings hehe
[02:16:30] <trentster> heh cool
[02:17:38] <Harvi> yes watched the videos you did too :)
[02:20:26] <trentster> aah yes, I am overdue to do some new screencasts soon, its been awhile.
[02:21:19] <trentster> Was planning on doing one on Windows dataset creation the correct way, as a lot of people seem a bit confused how to do it. I have seen a lot of blog posts about it and no mention of using sysprep which is the most important part.
[02:28:14] <Harvi> ahhh sysprep havent used that in years!
[02:32:33] *** while1eq1 has quit IRC
[02:33:02] *** while1eq1 has joined #project-FiFo
[02:33:38] <noahmehl> trentster: how come you didn't use the pkgsrc for afp?
[02:33:41] <noahmehl> netatalk i mean
[02:34:37] <trentster> I dont think it existed back then when I did the setup
[02:34:47] <noahmehl> trentster: ok cool
[03:21:06] *** while1eq1 has quit IRC
[03:21:27] *** while1eq1 has joined #project-FiFo
[03:22:00] *** while1eq1 has quit IRC
[03:22:30] *** while1eq1 has joined #project-FiFo
[03:26:09] <killfill> Harvi: what are you thinking when you say PaaS?.. something like cloudfoundry?
[03:27:51] <killfill> would love to run cloudfoundry on fifo/smartos.. :)
[03:34:36] <Harvi> Hi KillFill I run a bunch of custom automation scripts that provide me a cloudfoundry like ability to deploy all my business apps on my vanilla Fifo/SmartOS/ZFS infrastructure i would eventually like to build out my scripts into a more enviroment with a GUI possibly hack it into FiFO and combine it with chef solo to do all the grunt automation work, i monitor the environment with a combination of Zenos and log scrapping scripts an
[03:35:17] <Harvi> pseudo elastic scaling like adding in more HAProxy Loadbalancers and Database Instances on the fly to meet app demand
[03:38:12] <killfill> sounds awesome, would love to help you out with that
[03:42:15] <Harvi> I will have most of my code up on Github towards the end of november I have give my business partner a copy of "the cathedral and the bazaar" he comes from a closed source background and im convincing him to allow me to opensource the code so that we can work with other like minded people
[03:46:25] <killfill> cool
[03:46:32] <trentster> Harvi: that sounds great
[03:46:52] <killfill> im doing some chef magic at work too.. but wanted to do something a little different
[03:48:40] <Harvi> What kind of things are you doing with Chef at the moment and are you using solo or enterprise?
[03:49:57] <killfill> im using chef-server
[03:50:10] <killfill> to deploy node stuff..
[03:50:28] <killfill> but mainly its about configuring reverse procies, load balancing, cache's etc, etc.
[03:50:43] <killfill> *proxies
[03:51:50] <killfill> fun stuff, but its not paas.. i have a personal goal to go that way.. :P
[03:52:27] <Harvi> well maybe we could look to share some ideas and possibly collaborate
[03:53:00] <killfill> greate.
[03:56:34] <Harvi> bootstrapcharlie at gmail dot com thats my email address
[03:59:19] <Harvi> Would love to tell you more about it drop me an email if either you or trentster are intrested it would be great if would could get it working seamlessly/integrated with fifo via a decent api interface
[04:11:32] <Harvi> right time to watch some breaking bad and get some sleep see you all later
[04:14:31] <killfill> byebye
[04:25:02] *** Harvi has quit IRC
[04:40:48] <killfill> greate.. gzip fails quitly when running out of space.. :S
[05:33:03] <trentster> where on your fifo zone
[05:33:35] *** noahmehl_ has joined #project-FiFo
[05:34:04] *** noahmehl has quit IRC
[05:34:05] *** noahmehl_ is now known as noahmehl
[05:51:05] *** conan_the_destro has quit IRC
[07:38:51] <MerlinDMC> morning
[07:40:40] <MerlinDMC> meh ... way too early
[08:04:21] *** noahmehl_ has joined #project-FiFo
[08:06:23] *** noahmehl has quit IRC
[08:06:23] *** noahmehl_ is now known as noahmehl
[08:57:45] *** alcir has joined #project-FiFo
[09:37:25] *** mgls has joined #project-FiFo
[10:08:58] *** wiedi has joined #project-FiFo
[11:52:07] <trentster> hey all, anyone around in this channel who can confirm a bug for me quick in dev?
[13:56:02] *** tomww has quit IRC
[13:57:25] *** tomww has joined #project-FiFo
[14:39:15] *** noahmehl has quit IRC
[15:21:20] *** jim80net has joined #project-FiFo
[16:20:24] *** Harvi has joined #project-FiFo
[16:35:47] *** _lb_ has joined #project-FiFo
[17:06:16] *** wiedi has quit IRC
[17:10:34] *** alcir has quit IRC
[17:45:39] *** _lb_ has quit IRC
[17:46:29] *** _lb_ has joined #project-FiFo
[19:29:52] *** saltsa_ has quit IRC
[19:36:49] *** saltsa has joined #project-FiFo
[20:52:57] *** mgls has quit IRC
[21:57:51] *** Harvi has left #project-FiFo
[22:32:05] *** tomww has quit IRC
[22:33:15] *** tomww has joined #project-FiFo
[22:53:24] <konobi> bah, wrong channel
[23:04:02] * nahamu wonders which channel would have been the correct one.
[23:45:46] *** Harvey has joined #project-FiFo
[23:45:56] <Harvey> Evening all